Table of Contents

Trying to deduplicate records with an LLM and finding it never gets every match right? That is expected, not a bug. When you use an LLM to match and merge duplicate records — the same customer entered five ways, issues that are really one issue, entities with slightly different spellings — you are doing probabilistic entity resolution, and it will never be 100% accurate. The mistake is to chase accuracy. The production question is not “how do I make the merge always correct?” (no answer) but “given that it won’t be, how do I run reliable work on top of it?” (answerable). The answer has three parts: deterministic blocking to narrow candidates, confidence routing to decide what humans see, and reversibility so any wrong merge can be undone.

Merging records is probabilistic entity resolution

Define it plainly: entity resolution is deciding whether two records refer to the same real-world thing. When an LLM makes that call from messy text — synonyms, translations, casing differences, minor typos — the decision is a probability, not a fact. So a fraction of decisions will always be wrong, in both directions: false merges (over-merge) and missed merges (under-merge).

Over-merge is the dangerous direction. Collapsing two things that should have stayed separate is destructive, and unless you design for it, irreversible. That single property — that the worst error is hard to undo — is what shapes the rest of the design.

Narrow candidates first with deterministic blocking

Before any probabilistic matching, cut the candidate space with structured metadata. The principle: partition records into blocks using structured keys, then match within a block using unstructured content. The structured key narrows candidates; the unstructured content drives the similarity judgment.

structured key (reliable metadata)  ->  blocking: who can match whom
unstructured content (text)         ->  similarity: do they actually match

This bounds both cost and error, because the LLM only compares records that already share a hard key. The catch: blocking quality is capped by key quality. A key with a clean canonical identity blocks well; a key with no canonical taxonomy or ambiguous values (free-text product names, vague dates) blocks badly. Choose blocking keys you can trust.

Route by confidence, don’t chase accuracy

Split merge decisions into lanes by confidence:

• High confidence → auto-merge. No human touches it.
• Medium confidence → merge but flag. Apply it, attach a “needs review” marker.
• Low confidence → do not merge. Keep it as a candidate link only.

The rule that matters most: never push a low-confidence decision into an automatic merge. A conservative merge bias — when unsure, don’t merge — keeps mistakes recoverable, because under-merge is cheap to fix later and over-merge is not.

Humans then become exception handlers, not inspectors. Auto-merges pass by default; people see only the flagged cases, sorted lowest-confidence first. Review load scales with the number of ambiguous cases, not the total volume.

Make every merge reversible

Confidence routing only works if mistakes are cheap to undo. Use append-only provenance: don’t overwrite a record on merge — append the source of every contribution, so any merged record can be split back apart by its source set. When you later find an over-merge, you unmerge by removing the offending contribution, and no data is lost because nothing was overwritten. Keep an append-only history of the classified contributions even after they fold in, so the lineage of what got merged stays auditable.

An irreversible merge system dies on first contact with a bad merge: one un-undoable mistake and people revert to full manual review, which kills the automation.

Merging is continuous, not one-shot

A merge is not a single batch decision. As the underlying record gains new information — a thread keeps growing, a later message reframes an earlier one — the merge should be re-evaluated. What looked like two issues may become one, or one may split into two. Design merging as an ongoing process keyed to updates, not a verdict you reach once.

Frequently asked questions

How do I deduplicate records with an LLM? Don’t treat it as a pure accuracy problem, because an LLM matching messy text is probabilistic and will never catch every duplicate. Structure the pipeline instead: narrow candidates with deterministic blocking (split records into blocks by a structured key, match within the block), route each proposed merge by confidence (auto-merge the clear ones, send the uncertain ones to a human), and keep every merge reversible. You are designing for the errors, not pretending they won’t happen.

Why can’t the LLM match duplicate records with 100% accuracy? Because it is judging whether two records point to the same real-world thing from noisy text — synonyms, translations, casing, typos — and that judgment is a probability, not a fact. A share of cases will always be wrong in both directions: false merges (over-merging) and missed merges (under-merging). Over-merging is the dangerous one, because collapsing two things that should have stayed separate is destructive and, unless you designed for it, irreversible.

What is deterministic blocking in entity resolution? It is the step before probabilistic matching where you cut the candidate space with a structured key. You split records into blocks by that key and only run fuzzy matching within a block. The structured key narrows candidates; the unstructured content drives the similarity judgment. The quality of blocking sets the ceiling for everything downstream — a bad key silently caps your recall.

How do I safely merge records when the match might be wrong? Two rules. Route by confidence so a human reviews the uncertain merges instead of everything, and make every merge reversible — append-only history, an unmerge path — so a wrong merge is a correction, not a catastrophe. When the worst error (over-merge) is recoverable, you can run the whole system at a confidence threshold instead of demanding perfect accuracy.

The takeaway

LLM merging can never be 100% accurate. So the production goal isn’t accuracy — it’s confidence routing plus reversibility, with deterministic blocking to narrow the field first.

The upstream reason merges are ambiguous in the first place — weak signals like negations and reversals getting dropped before the merge step — is covered in why summarizing first loses weak signals. For keeping source traceability while humans correct these records, see human-in-the-loop traceability for AI-generated records.