Table of Contents

AI extraction produces records you don’t fully trust, so humans correct them — deleting wrong ones, keeping good ones, adding the ones the model missed. The hard part is keeping traceability to the source while allowing those corrections. The pattern: preserve the AI output, layer human corrections on top, and model the source-to-output link as a first-class relationship object — not a hard foreign key — so you can sever a wrong link without destroying the corrected record.

Preserve the AI output; correct on top

Don’t mutate the AI’s record in place as if it were ground truth. Treat the AI output as a preserved layer and human edits as a correction layer. You keep both what the model produced and what the human decided, which gives you two things at once: an audit trail of AI-versus-human, and the ability to re-run or compare extractions later without having overwritten the original.

This is the same move as designing for reversibility in a merge pipeline: nothing the automation produces gets destructively overwritten by a human action. Corrections accumulate; they don’t erase.

Why a hard foreign key fails here

The obvious data model points each record at its source with a foreign key: an extracted issue references the email it came from. That works until a human decides the association itself is wrong — the issue was real, but it was linked to the wrong email. Now you want to cut the link while keeping the record.

With a hard foreign key, you can’t cleanly do that. Severing the key can orphan the record, violate a required relationship, or trigger a cascade that deletes the very correction you wanted to keep. The relationship and the record are entangled, so you can’t edit one without risking the other.

Model the relationship as a first-class object

Put the source-to-output link in its own object — a junction or bridge — that sits between the source and the record. The relationship becomes data you can add, flag, or remove independently of the two things it connects.

[ source: email ]---<  relationship object  >---[ record: issue ]
                         (add / flag / remove
                          without touching either end)

The platform-agnostic principle is: make the relationship a first-class object. Then “this record came from that source” is an editable, removable fact, not a structural dependency. Two consequences fall out for free: one record can carry several source links (one issue surfaced from several emails), and you can remove any one of them without affecting the others — or the record.

Keep corrections append-only

When a human corrects a record, append the correction with its own provenance — who, when, what changed — instead of overwriting. The lineage stays intact, so you can always reconstruct the AI original and the human-corrected state side by side. Combined with the relationship object, this means every fact in the system carries where it came from: the model, a specific source, or a named human edit.

The takeaway

Preserve the AI output, correct it with a human layer, and model the source link as a first-class relationship object — so you can cut a bad association without destroying the record.

This is the traceability half of running an AI extraction pipeline you can trust; the other half — deciding which AI outputs a human should even look at — is in running probabilistic entity resolution in production. For why the AI output needs correcting in the first place, see why summarizing first loses weak signals.