Write & query performance (Beta)

The honest version first: for raw indexed search and filtering, a specialized engine like Elasticsearch (or a native C++ columnar store) will be faster than Aito, and we don't try to beat them at it. Aito's job on this axis is to be fast enough to serve interactive queries while carrying a prediction layer those engines don't have. This page states where Aito actually lands — including a direct Elasticsearch head-to-head.

What we measure

• Query / search latency — filter and full-text queries over live data.
• Predict / recommend / relate latency — the inference layer; there is no equivalent in a search engine, so it's an Aito-only number, not a comparison.
• Write / ingest — how fast new rows become queryable.
• Scaling — how each of the above moves as the table grows (measured 1k → 100k today, with the 10M point to follow).

Query & search latency

Filter and full-text queries resolve out of the memory-mapped inverted index. Here is a direct head-to-head against Elasticsearch 8.15 on identical data and queries — the same 100k flat invoice rows loaded into Aito v1, Aito v2 and Elasticsearch, each queried through one shared HTTP client so client overhead is identical. Warm p50 latency (all three return the same hit counts):

Query	Aito v1	Aito v2	Elasticsearch
Exact filter (company = N)	10 ms	9 ms	3 ms
Full-text (description match)	7 ms	35 ms	3 ms
Filter + full-text	5 ms	6 ms	3 ms

Read it straight: Elasticsearch is fastest on every query — a couple of milliseconds, roughly 2–3× quicker than Aito on filters. That is the expected result and exactly the one we promised — a specialized search engine beats us on raw indexed search. Aito still answers in single-digit to low-double-digit milliseconds — interactive — over the same index it uses for prediction. (Aito v2's full-text row is the outlier; the rep2 engine's full-text scan is a known beta slow spot, tracked separately — v1's full-text stays fast.)

The point isn't the milliseconds, it's that Aito stays interactive on the search axis and carries the calibrated-prediction layer a search engine doesn't have — see the comparison that matters. These are single-machine committed baselines (Aito in-JVM, ES in Docker on the same host); read the ratio, not the absolute milliseconds.

Prediction latency — v2 vs v1 (the part search engines don't have)

This is where the milliseconds buy something a search index can't do at all — a calibrated, explained prediction over your live data, no model to train or serve. Here is the v2 (rep2 / CollectionDb) engine measured head-to-head against the production v1 (TableDb) engine on the 10M-row linked-invoice benchmark (predicting processor / acceptor / glCode), warm steady state:

Predict target	v2 accuracy	v2 latency	v1 accuracy	v1 latency
acceptor	0.88	~145 ms	0.88	~157 ms
glCode	0.72	~106 ms	0.81	~110 ms
processor	0.56	~426 ms	0.69	~202 ms

On acceptor, v2 already matches the mature engine's accuracy and is a touch faster; on glCode it's close on both; processor — a high-cardinality cross-table link prediction — is the one target where v2 still trails on both. That path runs the cross-table reexpression (whitening) step v1 has heavily tuned and v2 hasn't yet, so it's the concentrated, understood gap, not an engine-wide one. Each predict ranks hundreds of candidates and returns a $why factor tree.

Warm-up — first query, cold

The first query per target pays JIT, file-cache miss and dictionary first-touch. v2's memory-mapped columnar reads make cold-start markedly faster on the two lower-cardinality targets (no big heap structures to materialize on first touch); processor is again the exception:

Predict target	v2 first query	v1 first query
acceptor	~540 ms	~2009 ms
glCode	~317 ms	~973 ms
processor	~9108 ms	~6448 ms

Memory & disk

v2 is built lean, which matters as much as speed when packing many tenants on a box:

• Memory-mapped, zero-allocation reads — a lookup reads straight out of the mapped column file, not a heap-materialized structure, so the resident footprint for a given dataset is much lower than the row-oriented engine's.
• Compact on disk — a binary columnar format with adaptive byte widths; the linkage structure alone is ~2.6× smaller than the naïve layout.
• Cheaper to build — rebuilding a table's optimized state uses ~6× less working memory (and is ~2× faster) than the previous path.

Scaling

Predict latency stays sub-linear in the data: two orders of magnitude more rows cost well under a doubling of latency. Warm steady-state predict latency (mean across the processor, acceptor and GL-code targets) on the linked-invoice benchmark, measured at three scales:

Rows	Warm predict latency (mean of 3 targets)
1k	203 ms
10k	273 ms
100k	328 ms

From 1k to 100k — 100× the data — latency rises only a fraction: the columnar, memory-mapped layout amortizes per-query overhead better at higher row counts, so the cost of a prediction is dominated by the candidate ranking, not by scanning the table. That's why the gap to a row-oriented engine widens in Aito's favour as data grows rather than shrinking. Query latency scales with index size as expected.

These figures are a committed baseline measured on a single dev machine — read the shape (sub-linear), not the absolute milliseconds, which move with hardware. The numbers regenerate from the InvoicePerf harness at docs-build time (invoice-table-* metrics), so they can't silently drift. The 10M point and a paired v2-engine column are the next additions; the v2 predict path is currently heavier (its layered cursor stack is the active tuning target), so this table reports the mature v1 engine until the v2 multi-scale run is in.

Writes

Writes are incremental: inserted rows are queryable and usable as prediction evidence immediately — there is no separate reindex or model-retraining step (contrast the accuracy suites, where every ML baseline needs a full refit to absorb new data). Batched ingest uses a back-pressured write path with deferred segment merging so read latency stays stable under load.

Measured v2 ingest on the linked-invoice benchmark (batched 2,000-row commits through /api/v2):

Metric	v2 ingest
Throughput	21,716 rows/s
Per-commit latency (p50)	90 ms
Per-commit latency (p90)	107 ms

Deferred segment merging — the default write path — roughly triples ingest throughput versus eager merging (measured ~2.85× in the committed A/B) by batching the index rebuild instead of paying it on every commit.

Like the scaling table, these are single-machine committed baselines generated from the BookInvoiceIngestBench snapshot — read the magnitude (tens of thousands of rows/s, sub-100 ms commits), not the exact figure. Write throughput is markedly more sensitive to machine load than query latency: on a contended box the same ingest measures an order of magnitude slower, so the baseline is captured on an unloaded run.

The comparison that matters

Against a specialized search engine, expect Aito to be competitive but not category-leading on raw query speed, and incomparable on prediction — because the search engine simply doesn't predict. The evaluation you're really making isn't "Aito vs Elasticsearch on QPS"; it's "one system that searches and predicts over live data with no training pipeline" vs "a fast search engine bolted to a separately-trained, separately-served ML stack." On the first axis you trade some raw speed; on the second you remove an entire moving part.

Related

• Benchmarks overview · GL-coding accuracy · Clusters & scaling · Warming strategies