How big can my database get? (Beta)

How do accuracy and latency move as the table grows from a thousand rows to ten million? This page is the corpus-size ladder: one corpus, one build, both engines, five scales. (The other scaling question — how much labelled data do I need to start? — varies the training set instead, and is the section's lead page: how much data do I need?.)

It is the evidence behind the head-to-head on the Query, predict & write speed page, which states the 10M numbers and the resource footprint.

Every figure here is a build-time token resolved from committed InvoicePerf booktest snapshots — nothing on this page is hand-typed.

What happens to accuracy and latency as the table grows? The InvoicePerf harness answers that on one corpus family — linked synthetic invoices — at five scales, predicting the same three targets each time: processor and acceptor (links into a ~46k-row employees table) and GL code (a 32-class categorical field).

Two properties make this ladder worth publishing, and neither held for the version of this table we shipped previously:

• One engine vintage. Every point below was measured on the same build, so the curve shows the effect of data scale — not the effect of an engine change that happened between two measurements taken months apart.
• 511 held-out test invoices at every point, on both engines, instead of 63. At n=63 the standard error on an accuracy near 50% is ~6pt, the same size as the effect the curve is meant to show; at n=511 it is ~2pt.

One correction comes with them. The accuracy published here is end-to-end: correct answers ÷ all test queries. A metric we reported earlier divided instead by only those queries whose correct answer appeared in the candidate list at all — a different quantity, with a denominator that moves per engine and per scale. Both engines now retrieve the correct candidate in every test query at every scale, so the two coincide; candidate retrieval covers why that was not always true of v2.

Accuracy vs. rows — v1 (rep1 / TableDb)

GL code climbs from the very first decade of rows (40.7% → 66.9% → 78.7%). With only 32 classes, even a 1k-row table puts real evidence behind each one, and every further decade keeps adding — with diminishing returns above 100k.

The two link targets are flat until 1M, then take off. Processor sits at 14.9% / 15.5% / 16.0% across 1k → 100k and only moves at 1M (49.1%) and 10M (65.8%). That is a property of the corpus, not a measurement artifact: there are ~46k employees, so a 100k-row table gives roughly two invoices per employee — far too thin to learn a routing signal for any individual. The link-target scaling story is really a 1M-and-above story, and we would rather say so than draw a smooth line through the flat part. (Acceptor's dip at 10k, 17.2% → 14.5%, is ~1.6 standard errors at n=511 — noise plus the fact that each scale is tested on a different held-out slice, not a regression.)

The same ladder on v2 (rep2 / CollectionDb)

Read this ladder straight: from 100k up, v2 leads on both link targets and v1 leads on the categorical. At 1M, processor 49.1% (v1) vs 56.9% (v2) and acceptor 54.4% vs 76.1% go to v2, while GL code 74.4% vs 72.8% goes to v1 — and the same split holds at 100k and at 10M. v1 answers faster throughout (51 ms vs 111 ms at 1M).

This is a scaling ladder first: what it establishes is that both engines improve monotonically with rows. Read as a v1-vs-v2 verdict it is a split decision, not a win for either — and it is not the whole picture, because top-1 alone says nothing about how well-calibrated a prediction is.

Candidate retrieval — both engines now offer the full candidate set

Top-1 accuracy can only be as good as the candidate list it ranks, and this section used to document a real v2 weakness. v1 offered the whole company-scoped employee set regardless of whether it had ever seen that employee, while v2's candidate scan offered only employees it had actually observed in the training rows. At small scale that left v2 unable to return the right answer at all in most queries, and its apparently-strong small-scale accuracy was an artefact of dividing by a much smaller denominator.

That gap is closed: v2 now enumerates the full linked-table candidate universe. Retrieval is 100.0% at 1k, 100.0% at 100k and 100.0% at 1M, matching v1's 100.0% at every scale and on every target — so the table that used to sit here, tracking v2's convergence, no longer has anything to show.

This matters for how the numbers above should be read. Every accuracy figure on this page is end-to-end over all 511 test queries on both engines, so the comparison is like-for-like at every rung, and no v2 figure is flattered by a reduced denominator. v2's small-scale accuracy now moves for engine reasons only — at 1k its end-to-end processor accuracy is 14.3% against v1's 14.9%.

Mean rank falls too

Accuracy is a top-1 threshold, so a curve built on it can in principle move for threshold reasons alone. Mean rank of the correct answer is measured on a different footing and corroborates the same story independently:

Processor's correct answer moves from rank 31.6 to 6.5 out of a ~46k-employee candidate space, and GL code from 5.4 to 1.3 out of 32 — i.e. usually first or second. Note this improves through the 1k → 100k stretch where processor's top-1 accuracy is flat: the ranking is already getting better there, it just hasn't crossed the top-1 threshold yet.

Latency vs. rows — the sub-linear claim

This is the second story, and it holds cleanly now that it is measured on one vintage:

From 1 024 rows to 10 485 760 — four orders of magnitude — v1 predict latency goes from 23 ms to 160 ms. Four orders of magnitude of rows buy well under one order of magnitude of latency — about 6.9× — so the growth is roughly logarithmic in row count, not linear. v2 shows the same shape across the same range (37 ms → 304 ms, about 8.2×), with only the 10k rung not yet measured.

The mechanism is that a prediction's cost is dominated by ranking candidates, not by scanning rows — the memory-mapped columnar layout resolves the evidence for a candidate in time that depends on the candidate set, not on table size. The candidate set here (~46k employees, 32 GL codes) is fixed by the schema, so adding rows adds evidence per candidate rather than work per query. Read the shape; the absolute milliseconds move with hardware.

What this ladder does not say

• No cross-method comparison at scale. The RF / LightGBM / FLAML baselines on the GL-coding page exist only at the 10k-row point. Nothing here supports "the tree models plateau while Aito keeps climbing" — we haven't run them at 1M or 10M. That comparison is pending, and until it exists, the ladder is a statement about Aito alone.
• The test slice differs per point. Each scale is evaluated on the 511 rows immediately after its own training data (the harness requires the test stream to start where training ended), so this is a proper held-out setup but not a single fixed test set shared across N.
• Synthetic corpus, single seed, single machine. Absolute values are point estimates; the direction is the durable signal.
• v2 at 10M is missing, as noted above. An older v2 10M measurement exists and is discussed in the 10M prediction table, but it predates both the scoring redesign and the end-to-end accuracy fix, so it is deliberately kept out of this ladder rather than mixed into it.

Every figure above is a committed baseline from a single dev machine, regenerated from the InvoicePerf harness snapshots at docs-build time (invoice-{table,collection}-* metrics), so it can't silently drift.