Define record identity, group likely matches, compare material fields, and record an explicit decision for each conflict. Keep the originals and an audit of any approved merge or deletion.
In this guide
Which rows belong in the same duplicate group?
Row IDs distinguish source positions; they do not establish whether business content is duplicated. The last row is a separate test mutation, not a fourth original order line. Keep the original file and the decision log. Delete a repeated line only after deciding the actual business key and whether repetition has meaning in the source system.
| Row | Business content | Classification | Action |
|---|---|---|---|
| P01 | TRAY-01 / OAK / USD 18.00 | Reference for group G1 | Keep source row |
| P02 | TRAY-01 / OAK / USD 18.00 | Same business fields as P01 | Ask whether repeated source row may be removed |
| P03 | TRAY-01 / WALNUT / USD 21.00 | Different variant | Keep separate |
| P02 changed-price test | TRAY-01 / OAK / USD 19.00 | Conflict rather than exact repetition | Check authoritative quote and date |
Choose a key that represents the actual item
A product name is often a label rather than a reliable identity. Write which combination of supplier, SKU, variant, or source-system ID identifies the item. Keep that rule beside the import so another operator does not silently substitute a shorter key.
Distinguish exact repeated rows, different variants, and competing updates to the same record. Each category needs a different action. A newer file date also does not prove that every field in that file is more authoritative; retain field-level source details when sources disagree.
- Preserve source file and row identifiers, then create a candidate grouping key without overwriting the original columns.
- Group matching keys and compare variant, units, price, currency, timestamp, and supplier context within each group.
- Mark exact repeats separately from conflicts and legitimate multiple records.
- For a conflict, specify which source or business rule can resolve it. Keep unresolved rows out of an automatic merge.
- Record retain, merge, update, or remove as a reviewer decision, with the surviving ID and the fields affected.
- Reconcile counts after the operation and keep a mapping from removed or merged rows to the retained record.
Understand what the importer will do with duplicates
Inspect your application's documented update semantics before sending a candidate group into a merge. Grist's import documentation describes different handling for duplicate incoming keys and duplicate existing keys. A completed import can therefore leave a result that differs from your intended conflict-resolution rule.
Resolve or isolate the duplicate group in a review table first. Then inspect the import preview and count the records that should be added versus updated. Exporting a clean-looking table is not evidence that conflicting source values were reconciled correctly.
Produce a review queue with explanations
Classify these duplicate candidates using the stated IDENTITY RULE: [rule]. ROWS WITH SOURCE IDS: [paste] AUTHORITY RULES: [effective dates, supplier approvals, or unresolved] Return exact repeats, legitimate variants, conflicting updates, and uncertain matches. Explain the fields that distinguish each group. Propose a review action and retained ID where justified. Do not delete rows, choose a price, or merge identifiers from a name similarity alone. Preserve every input row ID in the report.
Keep the decision available for the next supplier file
Save the identity rule and resolved examples with the cleanup history. The next run should flag a familiar conflict under the same rule rather than quietly making a different choice. Keep the final exception list with the file you hand over.