Skip to main content
ScrapingBuddy
The journal
Data quality3 min read

Product matching: identifiers first, similarity second

Build a reviewed product crosswalk that respects variants, pack sizes and units before comparing offers across retailers.

ScrapingBuddy Editorial
In this article
  1. Define the level at which products must match
  2. Use reliable identifiers with their scope
  3. Normalise attributes without erasing meaning
  4. Use similarity to propose, not to conceal uncertainty
  5. Maintain the crosswalk as a versioned asset

THE KEY IDEA

A similar name is a candidate, not a match. Preserve the attributes that would make a price comparison invalid.

01Define the level at which products must match

Decide whether the task requires the same model, the same variant or an equivalent substitute. These are different relationships. A price-monitoring workflow often needs exact size, pack and configuration, while assortment research may intentionally group broader product families.

Name the relationship in the data model. Do not mix exact matches and substitutes in one unqualified matched column. The consumer should know whether it is comparing identical items or analysing a broader competitive set.

02Use reliable identifiers with their scope

Manufacturer identifiers can be strong matching evidence when present and valid. Retailer-specific SKUs usually belong to that retailer's catalogue and should not be treated as universal keys. Preserve the identifier type, original value and source.

Check for identifier conflicts and malformed values before accepting a match. An exact string match can still be wrong if a source has placed a parent-product code on every variant. Use variant attributes as an independent consistency check rather than assuming an identifier settles every question.

03Normalise attributes without erasing meaning

Standardise units and represent pack count separately from unit size. Keep original text for audit. A two-pack of 500 ml containers is not the same offer as one 500 ml container, even if their product titles share almost every word.

Preserve attributes such as colour, flavour, condition and configuration when they affect comparability. Do not remove all numbers from names as a cleaning step; those numbers may identify the model or pack size.

Illustrative matching decision
A: Example detergent, 2 x 500 ml
B: Example detergent, 1 x 500 ml
Family match: possible
Exact offer comparison: reject (pack count differs)

04Use similarity to propose, not to conceal uncertainty

Restrict candidates to relevant brands and categories before applying text similarity. Score the attributes that matter to the task and reject hard conflicts. A very similar title should not override a different model number or an incompatible size.

Route ambiguous cases for review and keep the decision with its evidence. A confidence score is only useful if its meaning has been evaluated. Avoid presenting an arbitrary similarity number as the probability that a match is correct.

05Maintain the crosswalk as a versioned asset

Store accepted and rejected matches, the rule version and review history. Recheck mappings when source identifiers or critical attributes change. A persistent crosswalk can reduce repeated work, but only if it has a clear invalidation policy.

Evaluate precision and coverage separately using a manually reviewed sample. High coverage achieved by accepting weak matches can damage a pricing workflow. Keep unmatched records visible so the business can decide whether to expand collection, improve attributes or accept a narrower reliable comparison.

References & further reading

Put it into practice.

Try a public-page workflow, or scope a maintained dataset with our team.

All articles