By Ihnat Zharikhin, Member of Technical Staff
Every invoice that arrives at Light gets read by an AI before a human ever sees it. A vendor emails a PDF. Within seconds a model pulls out the invoice number, the date, the amount, the currency, the payee, and every line on the page. The pipeline then decides the ledger account and the tax code. By the time a finance person opens it, the booking is already proposed. Most of the time they approve it.
That reading step runs on a frontier model API. It works, and we didn't set out to prove otherwise. The question was narrower, and to us more interesting: could we do it ourselves?
Most companies use endpoints from frontier labs. You then accept their roadmap, pricing and quirks (and suddenly models being unavailable, like Fable). We wanted to challenge this status quo rather than lazily accepting it.
Why bother
The bar was to match the accuracy we already have. Not beat it, match it. Our model has 8 billion parameters and runs on one rented GPU. The model it's up against runs in Google's data centres, estimated to be in the trillion-parameter class.
Here's what it would buy us.
Cost. Using our models versus paying for an API endpoint will yield 90% to 99% cost savings as processing scales. Replacing this one step doesn't delete the whole bill today: in testing it takes roughly ten percent off, since some other steps dominate the process in size. Ten percent is already a few thousands of dollars a month.
Time. Our reader takes 3.6 seconds at the median to read a document; the endpoint takes 18.4 seconds for the same task. Which is a 5x speed increase, from a model a fraction of the size, sitting next to the pipeline instead of across the internet.
Independence. Today, every invoice Light reads is a call to another company. That company decides when the model changes, when the old one is switched off, what it costs, and how fast we're allowed to call it. None of those are hostile acts; they're just decisions made by people whose priorities aren't ours. A model whose weights sit on our own disk doesn't change unless we change it.
Privacy. Having full control of the data in our own environment is a strategic priority for us at Light.
Optionality. With a fine-tuning pipeline we retain much more optionality. Open models improve every few months, and we're now positioned to take that improvement.
The labels were already in the database
The usual first obstacle to fine-tuning is that you have no training data. We didn't have that problem: the product has been generating it for years. The pipeline proposes an extraction. A finance person reviews and corrects it. The approved result is stored. That leaves every processed invoice as a PDF with a human-checked JSON object beside it: tens of thousands of documents from the last month, after filtering to customers whose agreements permit training.
Pixels, not text
An invoice PDF comes in two flavours. Most are digital, generated by someone's billing software, and carry an embedded text layer you can extract exactly. About 9% are scans: a photograph of a piece of paper, with no text in it at all.
The obvious architecture is to branch: extract text where you can, OCR where you can't, feed text to a language model. We didn't do that. We send the whole document, every page, as images, to a vision-language model, and we do no OCR at all.
The reasoning is that the information you need is often positional, not textual. Which of the four numbers on the page is the total? Usually the bold one in the box at the bottom right, while "Subtotal," "VAT," "Total," and "Amount Due" all sit within a few lines of each other. Which date is the due date? Frequently nothing but position tells you. The vendor's name is very often a logo, which is to say an image containing no text whatsoever.
Extracting text throws all of that away. It costs you layout, emphasis, alignment and proximity on 100% of documents, in exchange for perfect characters on the 91% that didn't need help. And it forces you to maintain two code paths that fail in different ways.
We picked an Apache-2.0 open-source vision-language model, and trained its 2B variant first to prove the approach cheaply, then the 8B as the one to ship.
Fine-tuning is the product, not the polish
We started with the small 2B model to test the idea cheaply. Stock, with a careful prompt describing exactly the JSON we wanted, it scored 53.1% across the five core fields. Then we ran a LoRA fine-tune (about fifteen minutes of GPU time, touching roughly 0.3% of the model's parameters) and the same model scored 85.5% on the same documents. The frontier API scores 87.4% on that set.
Fifteen minutes of training took a stock open model from barely usable to within two points of the system we were trying to replace.
What LoRA actually is
Fine-tuning a model normally means updating all of its weights. For an 8-billion-parameter model that means holding the weights, their gradients and the optimiser's bookkeeping in memory at once: tens of gigabytes, a multi-GPU machine, and a complete 16GB copy of the model at the end of it. Do that for three document types and you're storing three copies.
LoRA (Low-Rank Adaptation) freezes the original model entirely and trains a small pair of matrices alongside each of its weight matrices. The insight is that the adjustment a specific task needs is low-rank: it has far less internal structure than the full weight matrix it modifies, so it can be factored into two skinny matrices whose product has the right shape, while basically not losing the accuracy. Train those two, leave the eight billion frozen weights untouched, and you're optimising about 0.3% as many numbers.
The practical consequences are what matter:
- It fits on one rented GPU. Ours trained on a single 24GB card. For the 8B we used QLoRA, a variant that additionally compresses the frozen base model to 4-bit so it fits in memory alongside training; the frozen part doesn't need full precision because it isn't being updated.
- The output is tiny. An adapter is tens of megabytes, not sixteen gigabytes. You can version it in ordinary tooling and keep as many as you like.
- One base model can serve many adapters. Invoices, receipts and contracts can share a single loaded model on a single GPU, with the relevant adapter swapped in per request. This is a large part of why self-hosting is economical at our volume.
- It's cheap enough to be disposable. At fifteen minutes and a few dollars a run, retraining stops being a project and becomes a routine, which is exactly the property you want if the goal is not to be locked to any one model.
What LoRA does not do is teach a model something it fundamentally cannot do. It adapts capability that is already in there. Which turns out to be exactly what this task needs.
What that gap represents is not "the model got smarter." The stock model could already read the page
and emit valid JSON. What it could not do was know the conventions: that dates go out as YYYY-MM-DD
regardless of how they're printed, that amounts are integers in minor units, that "payee" means the
vendor and not the customer, that a credit note's sign convention is a particular way. Those aren't
capabilities. They're house rules. And a small model learns house rules from a few thousand examples
better than we expected.
The one lesson we'd pass on to anyone in the same position: if your task has stable conventions and you have examples of them, fine-tuning a small model is not a marginal optimisation. It closed roughly nine-tenths of the gap to a frontier model.
The cost of doing that deserves a plain statement, because it is the part most people get wrong about fine-tuning. The run behind the chart above took fifteen minutes and about ten dollars of rented GPU. Every run in this project put together, including the false starts, the out-of-memory failures and the experiments that went nowhere, comes to a few hundred dollars.
The numbers
The model we're shipping can be summarised in four numbers:
- 8 billion parameters: an open-source vision-language model, adapted with LoRA.
- 5,482 curated documents: invoices across fifteen languages, roughly a tenth of them scans.
- Zero vendor overlap: the split is grouped by vendor, so no vendor appears in both training and test and the model is never graded on an invoice template it has already seen.
- 820 held-out documents: the final test includes line items rather than headers alone.
We proved the approach on the small model, then trained the 8B and put it through that harder test. The result is the same: within about a point of the frontier model on the mean, comfortably inside the confidence intervals. A model running on one rented GPU reads these documents about as well as the API does.
Beyond those five fields the model also reads every printed row on the invoice (description and amount), which is what the rest of the pipeline needs in order to split a bill across accounts. We deliberately aren't quoting a score for that, because there is nothing honest to score it against: the only line-item labels in existence are the current model's own output, so any number would measure how closely we imitate it rather than whether either of us is right. What matters is whether those rows produce the correct booking, which is a question the next section can actually answer.
There are two things to know about that average:
- It holds across languages. English, Swedish, Danish, Dutch, German, French, Italian, Spanish, Polish and Portuguese all score in the same band. That mattered because our customer base is European and English is only about half the volume.
- It does not hold everywhere. We fall behind the baseline on scanned documents and invoices running to four pages or more. The total often sits on a late summary page that is harder to find in a long stack of images.
From reading to booking
Field accuracy is necessary, but it isn't what ships. What ships is a booked bill: ledger lines with an account, a tax code and an amount, resolved against one customer's chart of accounts by the steps that run after ours. A small difference in how a line is worded can send money to a different account, while a perfect invoice number changes nothing at all. The real question was therefore never whether our model extracts the same fields, but whether the pipeline it feeds still produces the same booking.
To find out we ran 238 held-out invoices through the actual backend, with the actual downstream steps, against those customers' actual chart of accounts, changing only which model reads the document. Alongside that we ran the unchanged pipeline a second time as a control, which turned out to matter more than anything else we measured.
Run twice over the same documents, the unchanged pipeline booked 29% of them differently. The system we were treating as a fixed reference is really a distribution. A model that reproduced its booking perfectly would still be marked wrong on roughly a third of documents, so no comparison against a single run of it can be read at face value.
Measured that way, we are not at parity with the incumbent: 59% against a 71% floor is a real twelve-point gap, and an earlier thirty-document run that suggested otherwise was too small to trust, so we withdrew it. But measured against the booking a human actually approved, which is the only arm in the test that isn't a model, we do as well as it does: 79 exact matches for ours and 80 for the incumbent on the same 235 completed invoices, about 34% each. Our disagreements with the incumbent aren't systematically worse bookings, in other words. They're different bookings that are correct about as often.
That last pair of numbers deserves a second look, because it isn't about us at all. The production pipeline reproduces the human-approved booking about a third of the time, rising to roughly two-thirds once you control for documents booked by older versions of the vendor model and for accounts a person re-assigned as a matter of policy. Most of that gap has nothing to do with extraction quality: nearly half of these bookings were changed by a person after the AI proposed them, for reasons that were never on the page in the first place.
Running it
The serving profile is compact:
- Memory: the model runs 8-bit quantised on a single 24GB GPU, using about 9GB.
- Quality: on the hardest test slice (every scan and every long document), aggregate accuracy was identical to full precision, with individual fields moving in both directions by noise.
- Latency: a document takes 3.6 seconds at the median, against 18.4 seconds through the API.
- Steady-state capacity: the box handles around 1,100 documents an hour against average demand of about 22.
A GPU is a fixed-width pipe where a hosted API is elastic, and we found the edge of that the hard way: when we threw all 238 parity documents at the box simultaneously it took 100 minutes, against 20 for the API. That is not a situation production produces. Invoices arrive a handful an hour, spread across the day, so nothing queues and the swap costs a document no measurable time at all; the 3.6 seconds above is what an invoice actually waits. A genuine burst would queue, though, and handling that gracefully (batching more aggressively, or bringing a second GPU up on demand) is something we're working on currently.
The economics are the part that improves with scale. Per document, running the model ourselves works out one to two orders of magnitude cheaper than a frontier API call, and the two costs behave differently as the business grows: the API bill rises with every invoice our customers send, while the GPU costs the same whether it reads a hundred documents a month or a hundred thousand.
Future steps
Next we do the same to the rest of our document processing. Receipts first, then contracts. Every lab model we replace with our own cuts cost and time by 3-10x, and we pass that straight to our clients: a safer, faster, cheaper product.
This model runs inside Light, the agentic accounting platform. See the product or book a demo.