Queries under writes (Beta)

Most benchmarks measure a database that has stopped. This one measures one that hasn't: a 1M-row linked accounting database being written to the whole time the queries run.

On that database an inference predict — the account of a posting, inferred from seven known fields including free text — costs 268 ms at p50 while the writer is paused, and 853 ms at p50 while a writer commits a new posting every 250 ms. That is 3.2× the idle figure — and the second number, not the first, is what decides whether a database stays usable while it ingests.

And at 3 points after that write load, the answer the live process served was byte-identical to what a cold engine computes from the same data. That check is the point of the page as much as the latency is.

The measurement

One run, one build, one machine. The two rows that matter are measured together, in alternating 5-second slices across 12 rounds, with the reader threads never stopping — only the writer pauses and resumes. So both figures see the same machine over the same minutes, and each predict is filed by whichever slice it started in.

Each p50 below is the median round, not the median of all queries pooled: rounds yield unequal sample counts, so pooling silently weights the faster ones.

The load the third row was measured under:

Correct while it writes

Latency under writes is easy to buy with a stale cache. So once the measured rounds are done and the database has absorbed every write of the run, the same run checks 3 times that it hasn't:

1. the writer pauses;
2. the live process answers a predict, using whatever it has cached after thousands of commits;
3. a copy of the store is opened in a brand-new engine with empty caches, and answers the same query from the data alone;
4. the two responses must match byte for byte — every probability, to the last digit.

These checks deliberately run outside the measured window. Each one copies the whole database and starts a second engine, and when they were interleaved with the timing rounds they visibly perturbed the latency they exist to validate.

Result: 3 of 3 checkpoints identical, with 0 failed queries and 0 failed writes.

This is a strict check on purpose. A cache that outlived the data it described, or a per-segment statistic reused after its segment was rewritten, would move a probability in the last decimal and pass every eyeball test. It fails this one.

It is a sample, not a proof: three answers out of hundreds served. What it rules out is a systematic staleness, not every conceivable one.

Why a write costs anything at all

A commit does not mutate the database in place — it mints a new state. The new state shares the old one's segments, except the tail, which is rewritten to absorb the new rows. That is what keeps writes cheap and readers undisturbed: a query already in flight finishes against the state it started on.

The cost is on the next query. Anything the engine holds per state is new-and-empty on the state a commit just created, and gets recomputed. The engine's job is therefore to key as much work as possible to something that survives a write — the segment's own content — rather than to the state:

• Segment columns decode once. A column's decoded form belongs to the segment blob, which is the same object before and after the commit, so an untouched segment is not re-decoded.
• Statistics are computed per segment and composed. The class counts behind a prediction, and the rows a linked attribute projects onto the main table, are built for one segment at a time and memoised under that segment's content key. After a commit, only the rewritten tail is rebuilt; every other segment's work is reused as it is.
• Merging segments does not re-key their samples. Composing per-segment evidence into a whole-table answer resolves each contributing position through a memoised ordinal map instead of rebuilding the mapping for the new state.
• What is genuinely per-state is bounded. The caches that must be keyed to a state retain a small fixed number of recent generations, so continuous writing cannot grow them without limit — which is what keeps the heap figure above flat across thousands of commits rather than climbing.

The second row of the table is what remains after all that: the first predict on a fresh state still re-pays the work that genuinely changed. The third row is the steady behaviour, where new commits keep arriving and readers keep hitting mostly-reusable segments.

What this does not say

One shape of query. The measured query is a seven-field inference predict — deliberately the expensive shape. A filter or full-text query is a different cost model, measured on Query, predict & write speed.

One scale, one node. 1M rows on a single machine. Nothing here says how the curve bends at 10M, and nothing here is a distributed result.

One write pattern. A single writer appending small commits, which is the common shape for a system ingesting a live feed. Bulk backfill is a different regime and has its own number, ingest throughput. Deletes, schema changes and explicit optimize are not in this run.

One build. The figures come from the engine as it stands on master, which may be ahead of the released build you are running. The exact commit that produced them is recorded in the metrics file this page reads (core/docs/metrics/writes-under-load.json), so a number here can always be traced to the code that produced it.

Latency moves with the box. These are wall-clock numbers from one shared dev machine, and the conditions are published with them. Rather than assume a quiet machine, the bench checks whether its own rounds agreed and publishes nothing when they did not — so a figure here is one the run demonstrated it could reproduce minute to minute, not one measurement that happened to land well. The ratio between the rows, and the correctness result, are the durable parts; the absolute milliseconds are not portable to your hardware.

A tail is a tail. The p90 and p99 columns pool individual queries rather than rounds, because a tail is a property of single requests. On a machine with neighbours, part of that tail is theirs.

Related

• Query, predict & write speed — query latency against Elasticsearch, prediction latency, and ingest throughput
• Memory & disk footprint — what a database of this size costs to hold
• How big can my database get? — the accuracy and latency ladder from 1k to 10M rows
• Warming strategies — controlling first-query cost