· By Sajeevan (Saj) Veeriah
Engineering software · 4 min read
The hard part of a CSV import is deciding what may change
Preserving identifiers, resolving ambiguity and making an import reviewable before it changes an operational system.
A CSV import can finish successfully and still damage the meaning of the data. An item code loses its leading zeros. A blank price becomes zero. Two similar descriptions are treated as the same product.
I built operator-reviewed imports and explicit matching rules into my pricing and inventory application. This article explains those design decisions through worked examples of product records and prices.
Define the file contract before the parser
RFC 4180 documents a common CSV format, including quoting fields that contain commas, line breaks or quotes. It is an informational RFC, and actual importers may impose their own rules. Record the destination's required headers, ordering, encoding and accepted values rather than assuming that any file labelled CSV is interchangeable.
Keep a distinction between a syntactically valid row and a valid business change. A parser can accept a negative quantity perfectly well. Whether that value is allowed depends on the field and the workflow.
Use a fixture containing an embedded comma, an embedded quote and a multiline description. Check the parsed field values and the exported round trip. Counting separators is not enough to validate a CSV parser.
Sources: [1]
Treat identifiers as identifiers
In a fictional catalogue, item 00127 is a string. Converting it to the number 127 discards information that may be required by another system. The same concern applies to barcodes and supplier references even when they contain only digits.
Define normalisation narrowly. Removing accidental outer whitespace may be appropriate; dropping punctuation or case may merge records that the source treats as distinct. Show the original and normalised values during review when that transformation affects matching.
When two rows normalise to the same code, surface the collision. Choosing the first row silently makes file order a business rule. A reviewer needs to know that a decision exists.
Make ambiguity visible
A useful matching sequence is exact approved identifier, then a maintained alias, then a suggestion requiring review. Similar descriptions can help a person investigate a mismatch; they are weak authority for overwriting an operational record.
The proposed review below keeps unresolved rows out of the approved change set. A missing supplier row is not automatically a request to delete an existing item.
| Incoming row | Proposed handling |
|---|---|
| Unique exact code; valid fields | Prepare a before-and-after comparison. |
| Known alias with an approved mapping | Show the alias used and the target record. |
| Similar description; different code | Require review; do not silently update. |
| Duplicate code or missing required price | Hold the row with a specific reason. |
| Existing item absent from the file | Leave unchanged unless a separate deletion rule is authorised. |
Sources: [2]
Make numeric rules explicit
SQLite's documentation explains that binary floating-point values are approximate. For monetary processing that requires exact decimal behaviour, choose an appropriate decimal representation or an integer representation with an explicit scale. Validate range and precision as part of that choice.
For an invented arithmetic example, AUD 10.00 with a 30% markup becomes AUD 13.00 before any separately specified tax treatment. A 30% margin calculation would instead divide the cost by 0.70, producing approximately AUD 14.2857 before rounding. Confusing the two rules changes the result far more than a display-format issue.
Specify when rounding occurs and how ties are handled. If supplier costs contain fractions of a cent, an integer-cents model alone may discard required precision. Keep this rule in the domain logic and test it independently of the screen.
Sources: [3]
A useful import has a before and an after
Before export, show changed, unchanged and held rows separately. Include the old value, proposed value and reason for the match. Give the reviewer a manageable set of decisions rather than a green button beside a large unexplained row count.
Test a second application of the same intended change. It should not create duplicate records or compound a markup. Also test a stale target catalogue: if another person changed the target after the comparison, require a fresh comparison or a defined conflict resolution.
A recovery export can help restore previous values, but its scope needs care. It should not overwrite legitimate edits made after the import. The useful acceptance question is whether an operator can explain each change and recover from a mistaken run without guessing.
Sources and further reading
Sources checked on 11 September 2026. Catalogue rows and AUD amounts are worked examples.