We Fine-Tuned an Open-Source 8B Model to Replace a 1T Model

Ihnat Zharikhin, Member of Technical Staff August 16, 2026 11 min read

Every invoice that arrives at Light gets read by an AI before a human ever sees it. A vendor emails a PDF. Within seconds a model pulls out the invoice number, the date, the amount, the currency, the payee, and every line on the page. The pipeline then decides the ledger account and the tax code. By the time a finance person opens it, the booking is already proposed. Most of the time they approve it.

That reading step runs on a frontier model API. It works, and we didn't set out to prove otherwise. The question was narrower, and to us more interesting: could we do it ourselves?

Most companies use endpoints from frontier labs. You then accept their roadmap, pricing and quirks (and suddenly models being unavailable, like Fable). We wanted to challenge this status quo rather than lazily accepting it.

Diagram of the payables pipeline. An invoice PDF goes to a reader, which turns the page into structured data. The rest of the pipeline then decides how to book it, producing a booked bill. Only the reader was replaced with a self-hosted model; everything after it is unchanged. — Reading the page is one step of several. It's the only one we touched.

Why bother

The bar was to match the accuracy we already have. Not beat it, match it. Our model has 8 billion parameters and runs on one rented GPU. The model it's up against runs in Google's data centres, estimated to be in the trillion-parameter class.

Cost. Using our models versus paying for an API endpoint will yield 90% to 99% cost savings as processing scales. Replacing this one step doesn't delete the whole bill today: in testing it takes roughly ten percent off, since some other steps dominate the process in size. Ten percent is already a few thousands of dollars a month.

Time. Our reader takes 3.6 seconds at the median to read a document; the endpoint takes 18.4 seconds for the same task. Which is a 5x speed increase, from a model a fraction of the size, sitting next to the pipeline instead of across the internet.

Independence. Today, every invoice Light reads is a call to another company. That company decides when the model changes, when the old one is switched off, what it costs, and how fast we're allowed to call it. None of those are hostile acts; they're just decisions made by people whose priorities aren't ours. A model whose weights sit on our own disk doesn't change unless we change it.

Privacy. Having full control of the data in our own environment is a strategic priority for us at Light.

Optionality. With a fine-tuning pipeline we retain much more optionality. Open models improve every few months, and we're now positioned to take that improvement.

The labels were already in the database

The usual first obstacle to fine-tuning is that you have no training data. We didn't have that problem: the product has been generating it for years. The pipeline proposes an extraction. A finance person reviews and corrects it. The approved result is stored. That leaves every processed invoice as a PDF with a human-checked JSON object beside it: tens of thousands of documents from the last month, after filtering to customers whose agreements permit training.

Pixels, not text

An invoice PDF comes in two flavours. Most are digital, generated by someone's billing software, and carry an embedded text layer you can extract exactly. About 9% are scans: a photograph of a piece of paper, with no text in it at all.

The obvious architecture is to branch: extract text where you can, OCR where you can't, feed text to a language model. We didn't do that. We send the whole document, every page, as images, to a vision-language model, and we do no OCR at all.

The reasoning is that the information you need is often positional, not textual. Which of the four numbers on the page is the total? Usually the bold one in the box at the bottom right, while "Subtotal," "VAT," "Total," and "Amount Due" all sit within a few lines of each other. Which date is the due date? Frequently nothing but position tells you. The vendor's name is very often a logo, which is to say an image containing no text whatsoever.

Comparison of two ways of reading an invoice. On the left, the page as an image: the total is identifiable because it is bold, boxed and bottom-right. On the right, the same page after text extraction: subtotal, VAT, total and amount due become four indistinguishable numbers in a flat stream, with nothing to say which one is the total. Layout loss affects every document, not only the nine percent that are scanned. — Extracting text costs you position and emphasis on every document, including the 91% whose characters were never in doubt.

Extracting text throws all of that away. It costs you layout, emphasis, alignment and proximity on 100% of documents, in exchange for perfect characters on the 91% that didn't need help. And it forces you to maintain two code paths that fail in different ways.

We picked an Apache-2.0 open-source vision-language model with 8 billion parameters and fine-tuned it for the task.

Fine-tuning is the product, not the polish

Stock, with a careful prompt describing exactly the JSON we wanted, the model scored 53.1% across the five core fields. Then we ran a LoRA fine-tune (about fifteen minutes of GPU time, touching roughly 0.3% of the model's parameters) and the same model scored 86.5% on the same documents. The frontier API scores 87.4% on that set.

Fifteen minutes of training took a stock open model from barely usable to within a point of the system we were trying to replace.

What LoRA actually is

Fine-tuning a model normally means updating all of its weights. For an 8-billion-parameter model that means holding the weights, their gradients and the optimiser's bookkeeping in memory at once: tens of gigabytes, a multi-GPU machine, and a complete 16GB copy of the model at the end of it. Do that for three document types and you're storing three copies.

LoRA (Low-Rank Adaptation) freezes the original model entirely and trains a small pair of matrices alongside each of its weight matrices. The insight is that the adjustment a specific task needs is low-rank: it has far less internal structure than the full weight matrix it modifies, so it can be factored into two skinny matrices whose product has the right shape, while basically not losing the accuracy. Train those two, leave the eight billion frozen weights untouched, and you're optimising about 0.3% as many numbers.

The practical consequences are what matter:

It fits on one rented GPU. Ours trained on a single 24GB card. We used QLoRA, a variant that additionally compresses the frozen base model to 4-bit so it fits in memory alongside training; the frozen part doesn't need full precision because it isn't being updated.
The output is tiny. An adapter is tens of megabytes, not sixteen gigabytes. You can version it in ordinary tooling and keep as many as you like.
One base model can serve many adapters. Invoices, receipts and contracts can share a single loaded model on a single GPU, with the relevant adapter swapped in per request. This is a large part of why self-hosting is economical at our volume.
It's cheap enough to be disposable. At fifteen minutes and a few dollars a run, retraining stops being a project and becomes a routine, which is exactly the property you want if the goal is not to be locked to any one model.

What LoRA does not do is teach a model something it fundamentally cannot do. It adapts capability that is already in there. Which turns out to be exactly what this task needs.

Bar chart comparing three systems on the same test set with the same scorer. The stock open model with no fine-tuning scores 53.1 percent, our 8B model after LoRA fine-tuning scores 86.5 percent, and the frontier API it is being compared against scores 87.4 percent. Fine-tuning closes almost the entire gap.

What that gap represents is not "the model got smarter." The stock model could already read the page and emit valid JSON. What it could not do was know the conventions: that dates go out as YYYY-MM-DD regardless of how they're printed, that amounts are integers in minor units, that "payee" means the vendor and not the customer, that a credit note's sign convention is a particular way. Those aren't capabilities. They're house rules. And a small model learns house rules from a few thousand examples better than we expected.

The one lesson we'd pass on to anyone in the same position: if your task has stable conventions and you have examples of them, fine-tuning a small model is not a marginal optimisation. It closed almost the entire gap to a frontier model.

The cost of doing that deserves a plain statement, because it is the part most people get wrong about fine-tuning. The run behind the chart above took fifteen minutes and about ten dollars of rented GPU. Every run in this project put together, including the false starts, the out-of-memory failures and the experiments that went nowhere, comes to a few hundred dollars.

The numbers

The model we're shipping can be summarised in four numbers:

8 billion parameters: an open-source vision-language model, adapted with LoRA.
5,482 curated documents: invoices across fifteen languages, roughly a tenth of them scans.
Zero vendor overlap: the split is grouped by vendor, so no vendor appears in both training and test and the model is never graded on an invoice template it has already seen.
820 held-out documents: the final test includes line items rather than headers alone.

On that harder test the result holds: within about a point of the frontier model on the mean, comfortably inside the confidence intervals. A model running on one rented GPU reads these documents about as well as the API does.

Beyond those five fields the model also reads every printed row on the invoice (description and amount), which is what the rest of the pipeline needs in order to split a bill across accounts. We deliberately aren't quoting a score for that, because there is nothing honest to score it against: the only line-item labels in existence are the current model's own output, so any number would measure how closely we imitate it rather than whether either of us is right. What matters is whether those rows produce the correct booking, which is a question the next section can actually answer.

There are two things to know about that average:

It holds across languages. English, Swedish, Danish, Dutch, German, French, Italian, Spanish, Polish and Portuguese all score in the same band. That mattered because our customer base is European and English is only about half the volume.
It does not hold everywhere. We fall behind the baseline on scanned documents and invoices running to four pages or more. The total often sits on a late summary page that is harder to find in a long stack of images.

From reading to booking

Field accuracy is necessary, but it isn't what ships. What ships is a booked bill: ledger lines with an account, a tax code and an amount, resolved against one customer's chart of accounts by the steps that run after ours. A small difference in how a line is worded can send money to a different account, while a perfect invoice number changes nothing at all. The real question was therefore never whether our model extracts the same fields, but whether the pipeline it feeds still produces the same booking.

To find out we ran held-out invoices through the actual backend, with the actual downstream steps, against those customers' actual chart of accounts, changing only which model reads the document. We graded both pipelines against the booking a human actually approved and paid.

The result is a tie: 79 exact matches for our model and 80 for the incumbent, on the same completed invoices. Our model books some invoices differently than the incumbent does, but its bookings are correct just as often.

Bar chart comparing end-to-end pipeline outcomes on the same invoices. The pipeline using the self-hosted 8B model exactly matches the human-approved booking on 79 invoices, while the incumbent pipeline matches it on 80. — End-to-end through the real pipeline, ours and the incumbent reproduce the human-approved booking equally often.

For the finance team, that tie is the headline. Light customers can process invoices faster and more securely, without sacrificing booking accuracy. Same bookings, five times faster, read on our own hardware in our own environment, so we retain far greater control over where our customers' financial data is processed. And as volumes grow, we can deliver more automation without escalating AI costs: one GPU costs the same whether it reads a hundred invoices a month or a hundred thousand. Most AI gets more expensive as you scale. Ours doesn't.

Running it

The serving profile is compact:

Memory: the model runs 8-bit quantised on a single 24GB GPU, using about 9GB.
Quality: on the hardest test slice (every scan and every long document), aggregate accuracy was identical to full precision, with individual fields moving in both directions by noise.
Latency: a document takes 3.6 seconds at the median, against 18.4 seconds through the API.
Steady-state capacity: the box handles around 1,100 documents an hour.

A GPU is a fixed-width pipe where a hosted API is elastic, and we found the edge of that the hard way: when we threw the whole test set at the box simultaneously it took 100 minutes, against 20 for the API. That is not a situation production produces. Invoices arrive a handful an hour, spread across the day, so nothing queues and the swap costs a document no measurable time at all; the 3.6 seconds above is what an invoice actually waits. A genuine burst would queue, though, and handling that gracefully (batching more aggressively, or bringing a second GPU up on demand) is something we're working on currently.

The economics are the part that improves with scale. Per document, running the model ourselves works out one to two orders of magnitude cheaper than a frontier API call, and the two costs behave differently as the business grows: the API bill rises with every invoice our customers send, while the GPU costs the same whether it reads a hundred documents a month or a hundred thousand.

Future steps

Next we do the same to the rest of our document processing. Receipts first, then contracts. Every lab model we replace with our own cuts cost and time by 3-10x, and we pass that straight to our clients: a safer, faster, cheaper product.