Query, predict & write speed (preliminary)

The honest version first: for raw indexed search and filtering, a specialized engine like Elasticsearch (or a native C++ columnar store) will be faster than Aito, and we don't try to beat them at it. Aito's job on this axis is to be fast enough to serve interactive queries while carrying a prediction layer those engines don't have. This page states where Aito actually lands — including a direct Elasticsearch head-to-head.

What we measure

• Query / search latency — filter and full-text queries over live data.
• Predict / recommend / relate latency — the inference layer; there is no equivalent in a search engine, so it's an Aito-only number, not a comparison.
• Write / ingest — how fast new rows become queryable.

Two neighbouring questions have their own pages: what the same database costs to hold is on Memory & disk footprint, and how all of it moves as the table grows is on How big can my database get?.

Query & search latency

Filter and full-text queries resolve out of the memory-mapped inverted index. Here is a direct head-to-head against Elasticsearch 8.15 on identical data and queries — the same 100k flat invoice rows loaded into Aito v1, Aito v2 and Elasticsearch, each queried through one shared HTTP client so client overhead is identical. Every query shape is warmed before any of them is measured, and the measured rounds interleave the shapes, so no shape carries the stack's warm-up ramp for the others. Warm p50 latency (all three return the same hit counts):

Read it straight: Elasticsearch is fastest on every query — roughly 1.2–1.7× quicker than Aito across the three shapes. That is the expected result and exactly the one we promised: a specialized search engine beats us on raw indexed search. Aito answers every shape in single-digit milliseconds — interactive — over the same index it uses for prediction.

The point isn't the milliseconds, it's that Aito stays interactive on the search axis and carries the calibrated-prediction layer a search engine doesn't have — see the comparison that matters. These are single-machine committed baselines (Aito in-JVM, ES in Docker on the same host); read the ratio, not the absolute milliseconds.

Prediction latency — v2 vs v1 (the part search engines don't have)

This is where the milliseconds buy something a search index can't do at all — a calibrated, explained prediction over your live data, no model to train or serve. Here is the v2 (rep2 / CollectionDb) engine measured head-to-head against the production v1 (TableDb) engine on the 10M-row linked-invoice benchmark (predicting processor / acceptor / glCode). Both columns are now the same engine vintage measured the same way, so accuracy is shown for both; the scaling curve behind it is the ladder. Latency is the mean per-predict time across the evaluation run (it includes the one cold first query — broken out separately below):

What this table says, and what it does not. The engines separate by target, and the separation narrows as rows grow. v2 is ahead on processor — the highest-cardinality link target, the ones that resolve a reference to a row in another table — at every rung from 1k to 10M in the ladder, but by less than the 5-point threshold below at every rung. On acceptor, the other link target, v2 leads through the middle of the ladder, the engines converge at 1M, and v1 edges ahead at 10M (within noise). On glCode, the 32-class categorical, they are not distinguishable at any scale above 1k.

Read individual cells against the sampling interval, not as exact rankings. Accuracy is deterministic — re-running the whole ladder on the same build reproduces every cell — but each cell is 511 held-out invoices, so a gap under roughly 5 points is not evidence of a difference between the engines. The only difference that survives that test is v2's acceptor lead at 10k–100k (at 1k it sits right at the threshold).

An earlier version of this page withheld the v2 accuracy column, because the v2 10M snapshot was then a different engine vintage from the v1 column and computed over a different denominator — printing them side by side would have compared two measurements of two builds. Both halves of that objection are now gone: every cell above and in the ladder comes from one build, one corpus and the same end-to-end denominator.

What the latency columns support at 10M rows: v2 is now faster on the categorical and slower on the links. It wins glCode (123 ms vs 154 ms — a ~20% gap, well outside the ~8% noise band), and trails on both link targets, though by less than it used to: acceptor (212 ms vs 106 ms, ~2×) and processor (284 ms vs 239 ms, ~1.2×). Those cross-table links run through rep2's layered cursor stack, which is the active latency-tuning target. This is the trade stated plainly: v2 buys link-target accuracy with steady-state latency, and wins the cold start instead. Each predict ranks hundreds of candidates and returns a $why factor tree.

Warm-up — first query, cold

The first query per target pays JIT, file-cache miss and dictionary first-touch. v2's memory-mapped columnar reads make cold-start faster on every target — no big heap structures to materialize on first touch. The gap is widest on the high-cardinality processor target (~2.3×), where both engines pay several seconds cold; on acceptor and glCode it is ~1.5×. These are single first-query measurements, so read them as an order of magnitude, not to the decimal:

Memory & disk — on its own page

The same 10M state, measured for footprint rather than speed, says v2 needs 3,258 MB on disk against v1's 7,617 MB — 2.3× smaller. Heap is lower too, by 3.8× in the configuration that was measured, but heap moves with configuration where disk does not, so both figures and the configuration behind them live on their own page:

→ Memory & disk footprint

Scaling

Accuracy and latency both improve with rows, and the full ladder — five scales, both engines, one build — now has its own page:

→ Scaling: accuracy & latency vs. rows

The short version: both engines improve nearly monotonically with rows, v2's lead on the link targets is largest in the middle of the ladder and closes by 10M, the 32-class categorical is a tie throughout, and per-query latency grows far slower than the table does. The 10M end-point of that ladder is the head-to-head above.

Writes

Writes are incremental: inserted rows are queryable and usable as prediction evidence immediately — there is no separate reindex or model-retraining step (contrast the accuracy suites, where every ML baseline needs a full refit to absorb new data). Batched ingest can use a back-pressured write path with deferred segment merging, which keeps write latency stable under load; it is opt-in (AITO_WRITE_DEFER_MERGE=true), and the server merges segments eagerly by default.

Measured v2 ingest on the linked-invoice benchmark (batched 2,000-row commits through /api/v2):

Deferred segment merging materially raises ingest throughput versus eager merging by moving the index rebuild off the request that writes. It is opt-in: set AITO_WRITE_DEFER_MERGE=true on the server; without it, segments are merged eagerly on each commit. (The committed ingest snapshot measures deferred mode only; a tokenized eager-vs-deferred A/B is pending, so no fixed multiplier is quoted here.)

Like the scaling table, these are single-machine committed baselines generated from the BookInvoiceIngestBench snapshot — read the magnitude (tens of thousands of rows/s, sub-100 ms commits), not the exact figure. Write throughput is markedly more sensitive to machine load than query latency: on a contended box the same ingest measures an order of magnitude slower, so the baseline is captured on an unloaded run.

The comparison that matters

Against a specialized search engine, expect Aito to be competitive but not category-leading on raw query speed, and incomparable on prediction — because the search engine simply doesn't predict. The evaluation you're really making isn't "Aito vs Elasticsearch on QPS"; it's "one system that searches and predicts over live data with no training pipeline" vs "a fast search engine bolted to a separately-trained, separately-served ML stack." On the first axis you trade some raw speed; on the second you remove an entire moving part.

Related

• Memory & disk footprint — what the same database costs to hold
• How big can my database get? — the 1k→10M ladder
• Benchmarks overview · GL-coding accuracy · Clustering (preview) · Warming strategies