We Made Our Agent Dumber. It Started Learning.

By Nobel Suhendra, member of technical staff

We rebuilt how Astra, our always-on agent, learns from a company's books. The obvious way to describe that work is "we gave our agent memory." It would also be wrong. Astra had memory. It wrote daily observations. It ran an introspection loop that noticed operational patterns and wrote down what it found. If you had asked, before any of this started, whether Astra learned from a company's financial records, the honest answer was yes.

What we changed was not whether it learned. It was how it came to believe things.

What the old loop was actually doing

The old mechanism was a free-text introspection loop. An agent reflected on whatever it happened to notice, then emitted blocks of prose into a large memory field, tagged like [astra:pattern] and stamped with a line reading Confidence: 0.8.

Nothing in the system parsed that number. Nothing tied the claim to the records that supposedly supported it. Every one of those [astra:pattern] blocks was proto-normative: rule-shaped and asserted with confidence, but never grounded in a count. The loop treated them as finished rules when they were only ever candidates.

Three failures fall out of that, and they compound.

Put those together and you have an epistemology, not a missing feature. It also happens to be a familiar one: notice what stands out, feel confident in proportion to how vivid it was, never audit the base rate. That is roughly how a person forms impressions about a business after a few weeks of looking at it.

You can make the noticing sharper forever and it will never be enough. A rule like "invoices always issue from this entity" needs an exact denominator, and the loop was never counting anything — there is no version of "notice harder" that turns a vivid hunch into a count.

The fix was a dumber agent, not a smarter one

The replacement is a pipeline that sweeps everything, in a fixed order, with the interesting judgments pushed as late as possible.

The shape we borrowed for it is an old one. DIKW is a classic hierarchy: data, then information, then knowledge, then wisdom. We use the first three rungs and leave wisdom to the philosophers. Data is the raw records. Information is a faithful compressed restatement of them. Knowledge is what survives verification: insights and rules that carry their evidence with them.

That sounds like a diagram on a slide, and it would have been, except that it settled arguments. Each rung has one job, and each transition between rungs has one owner. Restating is not interpreting. Interpreting is not verifying. When a claim comes out wrong you can say which rung produced it, which was never true of a loop that did all three at once inside a single prompt.

Stage one compresses every row into short factual statements. This stage does no judgment, no inference, no ranking. It will write "£42,300 paid to a vendor across 11 invoices." It will not write "this vendor is a key dependency," because that is an interpretation, and interpretation belongs to a later, named stage that owns it.

Keeping that stage stupid turned out to be load-bearing. We run it on a small model with thinking switched off. At the volume it processes, thinking was expensive — but the real cost was accuracy. The model started improving the text, and improvement is interpretation: a stage that sees one row at a time has nothing to check it against, so the polished version was usually just wrong. The dumb stage has to stay dumb.

Everything downstream reads that one shared information layer, which is what killed the salience problem. Coverage stopped being something an agent decided and became a property of the pipeline.

Then we added the piece that mattered most. Every claim has to cite the records it rests on. A cheap deterministic pre-check fails anything with no valid citation before a model sees it, and then a verifier from a different model family re-rates each surviving claim against only its cited sources. The verifier's number becomes the confidence. The generator's opinion of its own work is discarded. In testing, that changed the outcome without changing a word of the prose: nothing about the writing got better, the claims simply got attached to their evidence.

Faithful is not the same as complete

Here is the idea that shaped the rest of the architecture, and it took us a while to see it clearly.

The information layer is faithful. It never invents or distorts. It is also extremely lossy: roughly 0.036 statements survive per row, so about 96% of rows are discarded at compression time. Faithful and complete are different properties, and the two kinds of thing we want to learn sit on opposite sides of that line.

Descriptive insights tolerate loss. "Revenue is roughly 70% concentrated in two customers" is exactly the right shape for a human building a mental model. Approximation is the point. These climb the ladder in one direction, data to information to knowledge, and stop. They never need the raw rows again.

Normative insights do not. A rule an agent acts on is only trustworthy with its exact track record, and that count was thrown away during compression. No better extractor recovers it, because completeness was lost by construction rather than by accident. So norms travel a V shape through the ladder: up to the information layer to generate a candidate rule cheaply, then back down to the raw records for the exact count. The information layer is a hypothesis generator on that path, not a source of truth.

That single asymmetry generated most of the design. Two stores, because the two kinds of claim are consumed differently. Two consumption paths. And a deterministic aggregator that walks the rows and computes the ratio itself, because the whole point of descending is to get a number nobody guessed.

Where we stopped asking the model

Once you start looking, the design turns out to be a list of places we took discretion away.

1. The extractor does not judge. It restates rows as short facts and nothing more.
2. The aggregator has no model in it at all. The computed ratio is the confidence.
3. The synthesizer must use typed shapes. It may propose any pattern over any field, but each shape has exactly one deterministic evaluator over raw rows, which turns scoring into a finite dispatch instead of text-to-SQL chaos.
4. The proposing model is bounded to a published catalog of fields. Open-ended can never mean inventing a column.
5. Range bands are computed from the population, not chosen by the model, because scoring a model against its own guess is circular.

Open-ended in what gets proposed. Closed-form in how it gets checked.

The reversals

The interesting part of the rebuild was watching production data correct us, usually by reproducing the old loop's failure inside the new system.

One agent holding a contradiction. Our first descriptive synthesis was a single running reviser: one agent that read the new information and rewrote the whole insight set each run. It silently reworded stable claims, and worse, it forked. A single transaction of $64,692 got written up as both a customer payment and an expense, and the reviser held both claims happily, because nothing in the loop was responsible for reconciling them. We split it into a generator that only proposes and a consolidator that folds candidates into the living set through explicit lifecycle operations. The consolidator now owns contradictions and resolves them in the rows: it re-reads the handful of records behind the conflicting citations, drops the wrong claim, and says why, citing the row. Untouched insights carry forward byte-identical, so a stable claim stops drifting.

Trusting one clock. The scheduler tracked the latest business date it had seen and re-read a trailing window each run to catch stragglers. Real data broke it in two directions at once. Records edited or back-dated beyond the overlap were silently missed, and information written over a few wall-clock days carried business dates spanning years. Neither clock alone was sufficient, so extraction became two passes: a catch-up pass with no model in it that scans by system time to find what changed and marks three-day business-date slots dirty, then a period extract that re-reads each dirty slot by business date.

Rules that quietly evaporated. Norms were decaying across incremental runs: a rule found once would vanish on the next run that did not rediscover it. Fixed by routing norms by record type and persisting them across runs.

What teaching actually meant

The system that came out of it is less free than the one that went in. It does not decide what is worth noticing. It does not rate its own work. It does not estimate anything it could count. Most of the engineering was removing discretion from places where discretion was quietly producing confident, unverifiable claims.

That is the part I would take to a different stack entirely. Teaching an agent to learn was mostly a matter of deciding where it was not allowed to think.

See how Astra, our always-on analyst, works inside Light, the agentic accounting platform. See the product or book a demo.