AI Utility Bill Extraction Accuracy: How It Fails, and How to Catch It

Vision models read utility bills better than OCR-plus-regex ever did. That is not in dispute, and it is not the interesting part. The interesting part is that the errors changed shape: OCR failed loudly, producing 1,2SO kWh that any type check rejects, while a vision model produces 1,250 when the bill says 1,350 and hands it to you with no indication that anything happened. Accuracy on a field-by-field spot check looks excellent. The dataset can still be operationally wrong.

The useful thing about these failures is that they are systematic. They cluster around the same handful of bill structures, which means you can write checks for them instead of re-reading every bill.

Why the errors look different from OCR errors

Character-level OCR errors are out-of-distribution by construction. A garbled account number has a letter in it. A garbled amount has two decimal points. You reject it with a format check and move on.

A language model doesn’t garble. It resolves ambiguity, and it resolves it toward whatever is most plausible given everything else on the page. When the bill is cleanly printed and unambiguous, that produces the right answer almost every time. When the bill is ambiguous (a faint fax-quality scan, a total printed in a box that also holds a late-fee notice, three numbers stacked in a column under one shared header), it produces a plausible answer: correctly formatted, in range, with no distinguishing signal.

That’s the core problem. Format validation, type checks, and range checks all pass. Everything downstream of them is checking the wrong thing.

The failure modes

Confidently wrong values

A Con Edison commercial bill prints the amount due inside a shaded box next to “Total amount due by Mar 6, 2025.” On a clean PDF this is trivial. On a rescanned copy where the box border bleeds into the digits, the model returns $3,417.28 where the bill says $3,418.28. One digit. Correct currency format, correct magnitude, correct rounding, and nothing to check it against unless you built something to check it against.

Accuracy on legible bills is high. The failures concentrate in the tail, and the tail is where your problem accounts live.

Right value, wrong field

Bills carry several numbers that are all “the amount.” Current charges. Previous balance. Payments received. Adjustments. Total amount due. On an account that pays in full every month, all of those collapse and grabbing the wrong one costs nothing, which is precisely why the error survives testing.

Then a bill arrives with a $1,918.42 prior balance and a partial payment. Now current charges and total due differ by hundreds of dollars, and whichever one you captured into your “cost” column silently becomes wrong for that month. If you’re using cost for $/kWh trending, total due poisons it. If you’re using it for AP, current charges underpays. Both are correct numbers off the bill. Neither is the one you meant. Which field belongs in which column is a decision worth making once, deliberately, and the field-by-field breakdown is the place to make it.

Multi-meter bills flattened into one row

The single most damaging error, because everything reconciles.

PG&E bills one account across multiple Service Agreement IDs. Georgia Power offers Summary Billing to commercial customers with ten or more eligible accounts, which rolls them onto a single Southern Company summary bill. National Grid’s Massachusetts summary billing does the same for multiple electric accounts. In every case, page one shows a summary and the per-meter detail lives further back.

A model that reads the summary returns a perfectly correct total. It just returns one row where there should be three. Total due matches, current charges match, the service period matches, and the aggregate usage is the true sum. Nothing fails. You have simply lost the ability to see anything at meter level, which is where the actual signal is. If you are not sure whether the identifier you captured is an account, a meter, or a service agreement, that distinction is not cosmetic.

Date confusion

Three separate variants, all common:

Statement date captured as service end. A Con Edison bill shows “Billing period: Jan 15, 2025 to Feb 13, 2025” near the top and a statement date of Feb 14 near the account block. Guess which one ends up in service_end_date when the billing period is printed in small type inside a usage graph.

Overlaps and gaps between consecutive bills. Utilities move read dates. A meter read on Feb 13 and the next on Mar 16 is a 31-day period; miscapture either end and you get a 28-day period followed by a 34-day one, and your normalized usage-per-day series develops a sawtooth that looks like real behavior.

Periods spanning a month boundary. Nearly all of them do. Any pipeline that assigns a bill to a calendar month by looking at the statement date will attribute February consumption to March about half the time.

Unit assumptions

Gas gets billed in therms, CCF, or Mcf depending on the utility. Water gets billed in CCF, kGal, gallons, or cubic meters depending on the municipality. The bill states the unit, usually. Models fill in the common one when it doesn’t.

The magnitudes vary in how forgiving they are. CCF read as therms is roughly a 2.5% error, small enough that it will never trip a threshold and will sit in your data permanently. CCF read as Mcf is 10x. CCF of water read as gallons is 748x. kGal read as gallons is 1000x. Ameren and Xcel bill gas in CCF in some territories and therms in others, on visually similar bill stock, which is exactly the setup that produces a silent unit flip when a service territory changes.

Electric has its own version: demand in kW landing in the usage field, or kVAh captured as kWh on a bill that shows both.

Estimated reads recorded as actual

Con Edison marks estimated readings. National Grid labels reads “Actual” or “Estimated.” Most extractions ignore the flag entirely because it isn’t a number.

Estimates are fine on their own. The problem is the true-up: two estimated months run low, then an actual read arrives and the third bill absorbs the difference. Your trend line shows a 40% spike that isn’t real, your anomaly detection fires, someone spends an afternoon on it, and the answer is that nothing happened. Capture the read type and the spike explains itself.

Summary read instead of detail read

Some bills print a summary of charges on page 2 and the itemized rate detail on pages 4 through 7. A model working under a page limit, or one that decided it had found what it needed, returns the summary and no line items. Totals are right, there is nothing to reconcile against, and the supply-versus-delivery split you needed for a rate analysis is gone.

Hallucinated tariff codes

Rate schedules have recognizable formats, and no two utilities agree on one. PG&E: A-1, A-10, B-19, B-10S. Con Edison: SC 2, SC 9 Rate I, with a space rather than a hyphen. Georgia Power: TOU-GSD-17. SCE: TOU-GS-3.

A model that can’t read the code cleanly will produce something with the right shape. SC 9 Rate II when the bill says Rate I. A-10 when it says A-10-TOU. SC-2 with a hyphen Con Edison doesn’t use. Every format check passes, because the output is formatted like a real tariff code. It just isn’t the one on the bill. The only checks that catch this are a lookup against the utility’s published schedule list, or a comparison against what that same account had last month.

A worked example where every field is correct

One PG&E account, 1234567890-1, three service agreements at a single property. Statement date Mar 12, 2025. Service period Feb 6 to Mar 7, 2025.

Current charges $4,805.64. Previous balance $1,918.42. Payments received $1,300.00. Total amount due $5,424.06.

Extraction returns: account 1234567890-1, service period 2025-02-06 to 2025-03-07, usage 26800 kWh, current charges 4805.64, total due 5424.06. Every field is exactly what the bill says. Arithmetic reconciles: 4,805.64 + 1,918.42 − 1,300.00 = 5,424.06.

The dataset is still wrong. There is one row where there should be three, so meter 1010203042 — the common-area meter running at $0.2317/kWh against $0.1701 on the main service — is invisible. Blended into the total it disappears entirely. In June, service agreement 5551234568 is closed out and the portfolio shows a 24% usage drop at that site with no explanation, because the row that would have explained it never existed. And the whole thing is unusable for meter-level Portfolio Manager setup, where each meter needs its own record.

No field-level accuracy metric catches this. The extraction was perfect.

The validation layer

Checks worth implementing, roughly in order of how much they catch per unit of effort.

Arithmetic reconciliation

Two identities, both cheap:

sum(line_items)                                        == current_charges   ± $0.02
prior_balance − payments + adjustments + current_charges == total_due       ± $0.02

Tolerance matters. Utilities round line items independently and a two-cent drift is normal. Fail hard above five cents.

Cross-bill continuity

This period’s start should equal the previous period’s end, or that date plus one day. Both conventions exist and utilities are not consistent about which they use, so accept 0 or 1 and flag everything else as an overlap or a gap. Honest warning: on utilities that shift read dates by two or three days routinely, this check is noisy, and a noisy check gets muted. Tune the threshold per utility or you will end up ignoring it, which is worse than not having it.

Usage sanity

Compare each read to the same month prior year and to a trailing average. Flag ratios beyond roughly 3x in either direction. Then add one refinement that pays for itself: flag ratios that land near a known conversion factor. A ratio near 10, 100, 748, or 1000 is almost never real consumption. It is a unit error or a misplaced decimal, and knowing which one it’s near tells you where to look.

Rate reasonableness

Derive $/unit and check it against a band. This is the strongest single catch-all, because it finds errors in usage or in cost without you having to know which. Build the band from that account’s own history rather than a national average. If this meter has run $0.16 to $0.19/kWh for eighteen months, flag anything outside $0.12 to $0.24. National figures are too wide to catch anything.

Completeness

Expected bill count per meter per period, compared to what arrived. Missing bills are the invisible failure: nothing looks wrong, there is just less of it, and a portfolio total that is quietly 4% light because six sites stopped billing will pass every other check on this list.

Duplicate detection

Key on account, meter, and service period, not on file hash. Hash dedup misses the common cases: a bill re-downloaded from the portal is a freshly generated PDF with a different timestamp, and a rescanned paper copy is a different file entirely. When the key matches but the totals differ, that is a rebill, not a duplicate, and it should supersede rather than be discarded.

Confidence and human review

Model-reported confidence is weakly calibrated for this task; a hallucinated tariff code can arrive with high confidence because the model is confident about the format. Route on evidence instead: anything that failed a check above, any field that changed when it usually doesn’t (rate code, meter number, service address), any account or meter appearing for the first time, and the top slice by dollar exposure. Be realistic about the ceiling. If you’re reviewing more than a small fraction of bills you have rebuilt manual entry with extra steps, and the honest answer for a small portfolio may be that the checks aren’t worth building at all.

Run these against a month of already-processed bills before you trust them. The first pass usually finds things that have been wrong for a while.