How do you deduplicate CRM data?

Define match rules, review candidate matches, then merge with a stated field-survivorship rule. Doing it without survivorship rules means the merge silently picks which values live, which is not recoverable.

Why do duplicates keep coming back?

Because cleanup addresses the stock and not the flow. Without entry controls — form matching, import checks, integration keys — the duplicate rate returns to its previous level within a quarter or two.

What fields should be used to match?

Email for contacts is high precision but misses shared and changed addresses. Domain for accounts is strong but breaks on subsidiaries and shared hosts. Most working systems combine several rules with different confidence levels rather than relying on one.

Is automatic merging safe?

Only for high-confidence exact matches. Fuzzy matches should queue for review, because a wrong merge destroys the losing record's history and generally cannot be undone.

Deduplication

Identifying and merging records representing the same person or company.

Why it is harder than it sounds

Duplicate detection is a judgement about identity, not a string comparison. Two records with the same company name may be separate subsidiaries; two contacts with different emails may be the same person after a job change. Every rule trades false merges against missed ones.

Merges are usually irreversible

A merge discards the losing record's field values and can break references held elsewhere. Treat the survivorship rules and the review threshold as the important decisions — the matching algorithm is the easy part.

Choosing rules

Common match rules and their failure modes

Rule	Precision	Fails on
Exact email	High	Shared inboxes; people who changed address
Email domain	Medium	Free-mail domains; shared hosting
Company name fuzzy	Low	Subsidiaries; common names; abbreviations
Website domain	High for accounts	Multi-brand groups; regional domains
External system ID	Highest	Only where an integration supplies one

Source: Assessment stated here

A working process

Measure the duplicate rate first, by object, so the effort is proportionate and the improvement is measurable.
Auto-merge only exact, high-confidence matches. Queue everything else for human review.
Write survivorship rules per field before merging anything: which record's value wins, and what happens to activity history and related records.
Merge in batches with a sample check after each, rather than in one pass. A rule error caught after fifty records is recoverable; after fifty thousand it is not.
Then fix the flow — form matching, import validation, integration keys — or the rate returns.

Prevention, in order of effect

Match on submission so a form fill updates an existing record rather than creating one.
Validate imports against existing records before they load, which is where large duplicate batches originate.
Use a stable external ID wherever an integration can supply one — it removes the judgement entirely.
Constrain manual creation with a search-first step, which costs seconds and prevents the most common single cause.

RELATED TERMS

Data Hygiene

The continuous practice of keeping records accurate, complete and current — as distinct from data quality, which is the state that practice produces.

Enrichment

Appending third-party data to records so routing, scoring and segmentation have fields to operate on.

Single Source of Truth (SSOT)

The declared authority for a given piece of data — the system whose value wins when two systems disagree. Correctly applied it is decided per field, not per system.

COMMON QUESTIONS

How do you deduplicate CRM data?: Define match rules, review candidate matches, then merge with a stated field-survivorship rule. Doing it without survivorship rules means the merge silently picks which values live, which is not recoverable.
Why do duplicates keep coming back?: Because cleanup addresses the stock and not the flow. Without entry controls — form matching, import checks, integration keys — the duplicate rate returns to its previous level within a quarter or two.
What fields should be used to match?: Email for contacts is high precision but misses shared and changed addresses. Domain for accounts is strong but breaks on subsidiaries and shared hosts. Most working systems combine several rules with different confidence levels rather than relying on one.
Is automatic merging safe?: Only for high-confidence exact matches. Fuzzy matches should queue for review, because a wrong merge destroys the losing record's history and generally cannot be undone.