Return Data Is Dirty: Building a Cleaning Pipeline That Scores Can Trust
Direct answer: Return scores fail most often because the underlying data is dirty, not because the model is weak. A cleaning pipeline that deduplicates events, reconciles SKU and order identities across systems, normalizes reason codes, and measures its own error rate will improve any scoring system more than a new algorithm will.
Teams usually debug return scoring by tuning the model. Then someone pulls raw return records and finds the truth: the same return logged by the warehouse and the carrier as two events, SKUs that changed mid season, reason codes from three different eras of the returns portal, and refunds recorded without the return they belong to.
A model trained on that is not inaccurate by accident; it is accurately learning a distorted record. Cleaning the pipeline is unglamorous, permanent work, and it is where most of the scoring accuracy a retailer will ever get is hiding.
Where returns data gets dirty
Returns touch more systems than almost any retail event: the commerce platform, the returns portal, the warehouse system, the carrier, the payment processor, and customer service tooling. Each generates its own record with its own identifiers and timestamps, and they disagree in small ways that compound into wrong customer histories.
Specific classics include returns double counted when a warehouse scan and a carrier scan both create events, exchanges recorded as a return plus a sale that inflates return rates, partial returns logged at full order value, and test and training transactions from staff that never get excluded.
Identity resolution is the foundation
Before any scoring, every return event must resolve to one customer, one order, and one item, consistently across systems. That means a mapping discipline for SKU changes and variants, order ID normalization between platforms, and customer matching that handles guest checkout, shared family accounts, and email typos without falsely merging strangers.
Errors here are not neutral. Falsely merged customers concentrate returns onto one profile and trigger abuse flags on an innocent person. Missed merges scatter an abuser's history across five profiles, each under threshold. Identity quality is scoring quality, in both directions.
Normalizing reasons, statuses, and time
Reason codes need a governed mapping: every historical and portal specific code folded into a stable taxonomy, with free text sampled regularly to catch new patterns the codes miss. Statuses need the same treatment, so that returned, in transit, refunded, and rejected mean one thing each across every source.
Time deserves equal care. Scoring windows and decay calculations break quietly when one system records ship date, another records warehouse receipt, and a third records refund date, all labeled date. Define the event timestamps your scores use and enforce them at ingestion, not at analysis time.
Measure the pipeline, not just the model
A cleaning pipeline should publish its own health metrics: duplicate rate, match failure rate, orphan refunds, impossible values, and the size of the unknown category in every taxonomy. When scoring accuracy shifts, these metrics tell you whether the world changed or your data did, and they usually answer faster than model diagnostics.
Build the unglamorous exports too: a sample of records a human can audit weekly, and an exceptions queue where unresolved events wait instead of silently polluting scores. Dirty data handled visibly is an asset; dirty data hidden by a confident model is a liability with a dashboard.
Related reading
Related reading: Return Reason Codes Are Lying to You: Reading What Customers Actually Mean, Return Scoring on Marketplaces: Score the Buyer and the Seller Separately, and Return Score Appeals: How to Give Good Customers a Second Chance.
Frequently asked questions
Why does return scoring underperform even with a good model?
Usually because the training and scoring data contains duplicates, mismatched identities, inconsistent reason codes, and mixed timestamps from multiple systems. A model learns the record it is given. Cleaning and measuring the data pipeline typically improves accuracy more than further model tuning.
What is the most damaging kind of dirty returns data?
Identity errors. False merges pile multiple people's returns onto one profile and produce wrongful abuse flags, while missed merges scatter one abuser across several profiles that each look harmless. Both directions of error destroy trust in the scores, with customers and internally.
How often should return data be cleaned?
Continuously, at ingestion, not as a periodic project. Deduping, identity resolution, and taxonomy mapping should run on every event as it arrives, with a weekly human audit of samples and exceptions. Periodic cleanups guarantee the data rots between them.
What metrics show a returns data pipeline is healthy?
Low and stable duplicate rates, a small percentage of events failing identity matching, few orphan refunds without returns, a small unknown share in reason code mapping, and timestamp consistency across sources. Publishing these alongside scoring accuracy keeps both honest.