// ai automation · 30 july 2026
“95% accurate”
is not a number.
It is the most quoted figure in document automation and the least useful. Here is what to ask for instead, and why the validation layer matters more than the model.
// the maths
Do the arithmetic on that claim.
Take a supplier invoice with ten fields you care about: invoice number, date, supplier name, GSTIN, three line items, taxable value, tax, total.
If each field is independently 95% accurate, the chance that a given document is entirely correct is 0.9510 — about 60%. Four documents in ten contain at least one wrong field. That is a very different proposition from what “95% accurate” sounds like when a salesperson says it.
Fields are not really independent, so the true figure is usually somewhat better. But the direction is right and the intuition matters: aggregate accuracy hides exactly the failures that hurt.
Worse, aggregates are usually dominated by easy fields. A system that reads the invoice date perfectly every time and the GSTIN correctly four times in five can still report a healthy-looking average while being unusable for the one thing you needed.
// what to ask for
Ask for the per-field table.
The number worth having looks like this — measured on your documents, not a benchmark set:
| Field | Accuracy | Failure mode | Caught by check? |
|---|---|---|---|
| Invoice number | 99.4% | Confuses similar serials | No — needs review |
| Invoice date | 99.1% | DD/MM vs MM/DD | Yes — range check |
| GSTIN | 97.8% | O/0 and I/1 confusion | Yes — checksum |
| Line item quantities | 96.2% | Merged rows | Yes — sum reconciliation |
| Total | 99.6% | Rounding | Yes — recomputation |
Now the picture is actionable. The fields with lower accuracy are mostly covered by a deterministic check, so an error is caught rather than absorbed. The one that is not covered — the invoice number — is the one that needs a human glance or a lookup against your purchase orders.
This is the table we produce during a paid pilot before anyone commits to a full build. It is a slower answer than a demo and a far more honest one.
// where safety comes from
The validation layer is the product.
The model is the least interesting part of a document automation system. What makes it safe is the deterministic code that runs afterwards and refuses to accept an implausible answer:
Re-computation. Add the line items. Recompute the tax. Compare against the extracted totals. If they disagree, the arithmetic wins and the document goes to review.Checksums and check digits. A GSTIN has a checksum. A container number has an ISO 6346 check digit. An IBAN, a PAN, an ISBN — many identifiers can be arithmetically verified, which turns a whole class of misreads into caught errors.Range and consistency checks. An invoice dated three years in the future is wrong. A due date before the invoice date is wrong. A quantity of 4,000 vials where every prior order was 40 is worth a second look.Precision preservation. This one is easy to miss. If a document prints a gross weight of18,240.500 kg, storing it as a float and echoing 18240.5 has changed the document. Trailing zeros carry meaning about measurement precision, and at a customs desk that is an argument. Carry such values as strings end to end.Confidence thresholds. Anything below threshold routes to a human queue where confirming takes seconds. Corrections feed the evaluation set, so thresholds tighten on evidence rather than optimism.// running a pilot
How to run a pilot that tells you something.
Use real documents, including the bad ones. The crumpled scan, the photo taken at an angle, the supplier whose layout changed last year. A pilot on your cleanest fifty documents measures nothing you need to know.Label a ground-truth set by hand. A hundred documents is usually enough to expose the systematic failures. Yes, this is tedious. It is also the only way to get a number rather than an impression.Measure per field, and separate caught from uncaught errors. An error a checksum catches costs seconds of review. An error that flows silently into your ledger costs an audit.Decide with the table in front of you. If the fields that matter are strong and the weak ones are covered by checks, proceed. If a critical field is weak and uncheckable, that use case is not ready — and a vendor who will not tell you that is not one to trust with your ledger.// summary
The short version.
Aggregate accuracy is marketing. Per-field accuracy on your own documents, with failure modes and which checks catch them, is engineering. Insist on the second.
You can see the shape of the output we produce, including the validation layer, in the worked examples on our AI automation page. If you have a document type you would like measured properly, send us a message.
// next step
Tell us what is actually broken.
A short message is enough to start. The first scoping call is free, and you will be talking to the person who would write the code.