Query, predict & write speed (Beta)

The honest version first: for raw indexed search and filtering, a specialized engine like Elasticsearch (or a native C++ columnar store) will be faster than Aito, and we don't try to beat them at it. Aito's job on this axis is to be fast enough to serve interactive queries while carrying a prediction layer those engines don't have. This page states where Aito actually lands — including a direct Elasticsearch head-to-head.

What we measure

• Query / search latency — filter and full-text queries over live data.
• Predict / recommend / relate latency — the inference layer; there is no equivalent in a search engine, so it's an Aito-only number, not a comparison.
• Write / ingest — how fast new rows become queryable.

Two neighbouring questions have their own pages: what the same database costs to hold is on Memory & disk footprint, and how all of it moves as the table grows is on How big can my database get?.

Query & search latency

Filter and full-text queries resolve out of the memory-mapped inverted index. Here is a direct head-to-head against Elasticsearch 8.15 on identical data and queries — the same 100k flat invoice rows loaded into Aito v1, Aito v2 and Elasticsearch, each queried through one shared HTTP client so client overhead is identical. Every query shape is warmed before any of them is measured, and the measured rounds interleave the shapes, so no shape carries the stack's warm-up ramp for the others. Warm p50 latency (all three return the same hit counts):

Read it straight: Elasticsearch is fastest on every query — roughly 1.2–1.7× quicker than Aito across the three shapes. That is the expected result and exactly the one we promised: a specialized search engine beats us on raw indexed search. Aito answers every shape in single-digit milliseconds — interactive — over the same index it uses for prediction.

The point isn't the milliseconds, it's that Aito stays interactive on the search axis and carries the calibrated-prediction layer a search engine doesn't have — see the comparison that matters. These are single-machine committed baselines (Aito in-JVM, ES in Docker on the same host); read the ratio, not the absolute milliseconds.

Prediction latency — v2 vs v1 (the part search engines don't have)

This is where the milliseconds buy something a search index can't do at all — a calibrated, explained prediction over your live data, no model to train or serve. Here is the v2 (rep2 / CollectionDb) engine measured head-to-head against the production v1 (TableDb) engine on the 10M-row linked-invoice benchmark (predicting processor / acceptor / glCode). Both columns are now the same engine vintage measured the same way, so accuracy is shown for both; the scaling curve behind it is the ladder. Latency is the mean per-predict time across the evaluation run (it includes the one cold first query — broken out separately below):

What this table says, and what it does not. v2 leads on both link targets — the ones that resolve a reference to a row in another table (processor, acceptor) — while v1 keeps glCode, the 32-class categorical. That split holds at every scale from 100k up in the ladder, so it is a property of the two engines rather than an artefact of one measurement. v1 remains several times faster per predict; that is the honest cost of v2's link-target accuracy, not a rounding difference.

An earlier version of this page withheld the v2 accuracy column, because the v2 10M snapshot was then a different engine vintage from the v1 column and computed over a different denominator — printing them side by side would have compared two measurements of two builds. Both halves of that objection are now gone: every cell above and in the ladder comes from one build, one corpus and the same end-to-end denominator.

What the latency columns do support: at 10M rows, v2 is not faster on any target. It is level on glCode (160 ms vs 161 ms — a 1% gap, well inside the ~8% noise band), and materially slower on both link targets: acceptor (355 ms vs 107 ms, ~3.3×) and processor (396 ms vs 213 ms, ~1.9×). Those cross-table links run through rep2's layered cursor stack, which is the active latency-tuning target. This is the trade stated plainly: v2 buys link-target accuracy with steady-state latency, and wins the cold start instead. Each predict ranks hundreds of candidates and returns a $why factor tree.

Warm-up — first query, cold

The first query per target pays JIT, file-cache miss and dictionary first-touch. v2's memory-mapped columnar reads make cold-start faster on every target — no big heap structures to materialize on first touch. The gap is widest on the two lower-cardinality targets (glCode ~14×, acceptor ~2.6×); on the high-cardinality processor target the engines are closer, both paying several seconds cold:

Memory & disk — on its own page

The same 10M state, measured for footprint rather than speed, says v2 needs 597 MB of heap against v1's 2,276 MB — about 3.8× leaner, and 2.3× smaller on disk. That is the number that decides how many tenants fit on a box, so it has its own page:

→ Memory & disk footprint

Scaling

Accuracy and latency both improve with rows, and the full ladder — five scales, both engines, one build — now has its own page:

→ Scaling: accuracy & latency vs. rows

The short version: both engines improve monotonically with rows, v2 leads both link targets from 100k up while v1 keeps the 32-class categorical, and per-query latency grows far slower than the table does. The 10M end-point of that ladder is the head-to-head above.

Writes

Writes are incremental: inserted rows are queryable and usable as prediction evidence immediately — there is no separate reindex or model-retraining step (contrast the accuracy suites, where every ML baseline needs a full refit to absorb new data). Batched ingest uses a back-pressured write path with deferred segment merging so read latency stays stable under load.

Measured v2 ingest on the linked-invoice benchmark (batched 2,000-row commits through /api/v2):

Deferred segment merging — the default write path — materially raises ingest throughput versus eager merging by batching the index rebuild instead of paying it on every commit. (The committed ingest snapshot measures only the deferred default; a tokenized eager-vs-deferred A/B is pending, so no fixed multiplier is quoted here.)

Like the scaling table, these are single-machine committed baselines generated from the BookInvoiceIngestBench snapshot — read the magnitude (tens of thousands of rows/s, sub-100 ms commits), not the exact figure. Write throughput is markedly more sensitive to machine load than query latency: on a contended box the same ingest measures an order of magnitude slower, so the baseline is captured on an unloaded run.

The comparison that matters

Against a specialized search engine, expect Aito to be competitive but not category-leading on raw query speed, and incomparable on prediction — because the search engine simply doesn't predict. The evaluation you're really making isn't "Aito vs Elasticsearch on QPS"; it's "one system that searches and predicts over live data with no training pipeline" vs "a fast search engine bolted to a separately-trained, separately-served ML stack." On the first axis you trade some raw speed; on the second you remove an entire moving part.

Related

• Memory & disk footprint — what the same database costs to hold
• How big can my database get? — the 1k→10M ladder
• Benchmarks overview · GL-coding accuracy · Clusters & scaling · Warming strategies