Vector Search & RAG (v2)

Aito v2 stores dense embeddings in a Vector column and ranks rows by similarity with the $nearest query operator. That gives you three things from one store:

• Semantic retrieval — find the rows whose meaning is closest to a query (the retrieval half of RAG).
• Hybrid search — combine the vector with BM25 full-text and a prediction ($p) in one query, so lexical, semantic, and contextual signals rank together.
• Semantic inference — use a row's nearest neighbours as evidence to predict its label, which helps most exactly where token features fail (paraphrase, and especially across languages).

By default you bring your own vectors — compute them with your own model (any sentence/text encoder) and supply them as arrays of numbers; Aito stores, indexes, and searches them. Search is exact nearest-neighbour, so results are reproducible and there is no index to tune. (Optionally, a Vector column can declare an embedder so Aito embeds text for you — see below.)

The workflow

1. Encode your text (or images, etc.) into vectors offline, with your model of choice. Keep the dimensionality fixed.
2. Define a Vector column with that dimensions and a similarity (cosine for direction, dotProduct for raw inner product).
3. Ingest each row with its vector as an array of numbers.
4. Query with $nearest — optionally filtered, thresholded, and limited.

1–2. Schema

PUT /api/v2/schema/products
{ "type": "collection", "columns": {
  "id":        { "type": "Int" },
  "name":      { "type": "Text", "analyzer": "english" },
  "category":  { "type": "String" },
  "embedding": { "type": "Vector", "dimensions": 384, "similarity": "cosine" }
} }

cosine columns are unit-normalised on ingest, so $similarity is the cosine in [-1, 1]. See Schema Design → Vector columns.

The table must be a collection ("type": "collection", as above): a Vector column on a v1 table (type: table on the v1 engine) is refused with a 400, because vector storage and $nearest are part of the collection engine.

3. Ingest vectors

Vectors ride along with the rest of the row (batch or stream). Ingest large corpora in several batches, each under the request-size limit below.

POST /api/v2/data/products/batch
[ { "id": 1, "name": "running shoe", "category": "shoes", "embedding": [0.02, -0.11, …] },
  { "id": 2, "name": "wool sock",    "category": "socks", "embedding": [0.07,  0.03, …] } ]

Optional: embed text automatically

Instead of bringing your own vectors, a Vector column can declare a pinned embedder and a from text column. Then you send text — at ingest and at query — and Aito calls the embedder to produce the vectors:

"embedding": { "type": "Vector", "dimensions": 384, "similarity": "cosine",
  "embedder": { "preset": "tei", "endpoint": "http://tei:8080/embed", "model": "intfloat/multilingual-e5-small" },
  "from": "name" }

Ingest rows with just the from text ({ "id": 1, "name": "running shoe" }) — Aito embeds name into embedding. Supplying the vector directly still works and skips the embedder. At query time, a text near/vector value is embedded with the same pinned model, so stored and query vectors share one space:

{ "where": { "$nearest": { "near": { "embedding": "cordless mouse" } } } }

preset is the endpoint wire format (openai / cohere / ollama / tei). The embedder is pinned to the column — changing the model means re-embedding.

Reading the schema back. GET /api/v2/schema/{table} reports a Vector column exactly as you declared it — type, dimensions, similarity, and for an embedder column its embedder and from — so the response can be PUT to another table to recreate it. The embedder sends no authentication header, so an endpoint that needs a key could only carry it in its URL; since a schema is readable with a read-only key, an endpoint with a query string or user info reads back as "endpoint": null, and a PUT of that is refused until you supply it.

4. Retrieve with $nearest

POST /api/v2/_query
{ "from": "products",
  "where": { "$nearest": {
    "near":  { "embedding": [0.01, -0.09, …] },
    "where": { "category": "shoes" },
    "having":{ "$similarity": { "$gte": 0.8 } },
    "limit": 10 } },
  "select": ["id", "name", "$similarity"] }

Rows come back ordered by $similarity descending. The nested where filters before scoring (so limit is the top-k among matches), having/threshold sets a minimum score, and limit caps the results. Full operator details: Query Reference → Vector similarity.

RAG retrieval quality

For retrieval-augmented generation, $nearest is the retrieval step: embed the user's question, fetch the top-k passages, and hand them to your generator.

Because search is exact, retrieval quality is entirely a function of your embedding model — Aito reproduces it faithfully. On the public BEIR SciFact benchmark (5,183 abstracts, 300 claims) with all-MiniLM-L6-v2, $nearest scores nDCG@10 ≈ 0.65, matching the published figure for that encoder. Swap in a stronger model and the retrieval numbers move with it.

Hybrid search: combine lexical, semantic, and predictive signals

Vector similarity is one signal, and it isn't always the right one. Which signal wins depends on the query:

• BM25 full-text ($search/@@) nails exact tokens — names, SKUs, codes, rare terms — the things an embedding blurs into its neighbourhood.
• Vector nails meaning — paraphrases, and answers that share no words with the query at all.
• Prediction ($p) adds context — personalization, popularity, the probability of an outcome — which neither text nor vector can see.

They're complementary, not competing. On BEIR SciFact, bucketing queries by how much exact-term overlap they share with the answer, BM25 recall@10 runs from 0.00 (no shared word — it literally can't retrieve) to 0.96 (high overlap), while the vector arm is far flatter (0.53 → 0.98): BM25 is an exact-match tool, the vector a semantic one. Over 300 queries, 76 were found only by the vector and 8 only by BM25 — each rescues cases the other misses. So the robust move is to combine them, which Aito does in one query.

Rank fusion — the hybrid query (scale-free default)

BM25 scores are unbounded; cosine is [-1, 1]. You can't just add them. The hybrid query sidesteps the scale mismatch with Reciprocal Rank Fusion — it fuses the two rankings, not the scores:

{ "from": "docs",
  "hybrid": {
    "match":   { "field": "content", "text": "wireless noise cancelling" },
    "nearest": { "embedding": "wireless noise cancelling" },
    "k": 60,
    "weights": { "match": 1.0, "nearest": 1.0 } },
  "select": ["title", "$score", "$bm25", "$similarity"], "limit": 10 }

It runs both arms, fuses over the union of their top hits (so a strong hit in either arm survives — the recall win), and returns $score (the fused rank score) plus each arm's raw $bm25/$similarity. weights down-weights a weaker arm; on a corpus where one signal dominates, equal-weight fusion can trail the stronger arm, and the weight is the fix.

Probabilistic blend — $multiply (composes with prediction)

Rank fusion can't compose with a $p. The alternative turns each signal into a probability lift and multiplies them in an orderBy:

{ "from": "products",
  "orderBy": { "$multiply": [
    { "$similarity": { "name": "wireless" } },
    { "$vectorIdf":  { "embedding": [0.01, -0.09, …] } } ] },
  "select": ["id", "name"], "limit": 10 }

• $similarity is BM25 as a lift (exp(θ·bm25), already calibrated at θ=0.33).
• $vectorIdf is the self-calibrating vector lift (1/p)^θ, where p is the cosine's rarity over the candidates — a "vector IDF", exactly BM25's (1/p)^0.33 form applied to similarity. The model-specific cosine scale is absorbed into p, so one θ (default 0.87) transfers across embedding models with no per-model tuning. ($vectorSimilarity is the raw-cosine lift and $vectorCosine the raw score, if you want those instead.)

The three-way blend — add prediction

Because $p is just another $multiply part, you can rank by P(outcome) · text-similarity · vector-similarity in a single query — a personalized semantic search. Rank product candidates (over a link) by purchase probability × how well the name matches × how close the meaning is:

{ "from": "impressions", "get": "product",
  "orderBy": { "$multiply": [
    { "$p":          { "$context": { "purchase": true } } },
    { "$similarity": { "name": "wireless" }, "theta": 0.33 },
    { "$vectorIdf":  { "embedding": [0.01, -0.09, …] }, "theta": 0.87 } ] },
  "select": ["name", "$p"], "limit": 10 }

This is where one store earns its keep: a product whose name shares no query word can still win — the vector matches its meaning and the shopper's history predicts a purchase — while a merely-lexical match without either sinks. Text, semantics, and prediction, resolved together in one orderBy.

Tuning — ranking vs calibration

Two knobs, and they're orthogonal:

• Ranking depends only on the per-arm theta ratio ($similarity's θ vs $vectorIdf's θ) — a plain query parameter. A global exponent on the product is a ranking no-op. Tune the ratio (or lean on the calibrated defaults) to order results well.
• Calibration (whether $score reads as a probability) depends on the overall scale. A pure ranking objective will happily choose over-confident values; if you need a calibrated P — to threshold, or to compose as an honest probability — fit a small logistic over the two signals offline (a once-per-corpus step; the query then just applies the weights).

Rule of thumb: hybrid (RRF) when you want a robust default with no tuning; $multiply when you want to fold in a prediction or need a composable score; $vectorIdf over $vectorSimilarity because it self-calibrates.

Semantic inference

Beyond retrieval, a row's nearest neighbours are strong evidence for its own label — a semantic k-NN classifier. This shines where surface form and meaning diverge and token features break down.

The sharpest case is cross-lingual. Trained on English and tested on other languages (Amazon MASSIVE intent classification, multilingual embeddings), a token classifier collapses toward the prior — the test words simply aren't in the training vocabulary — while nearest-neighbour voting in embedding space still lands the right intent:

In-language, semantics edges a strong token model; cross-lingual, it wins 4–5×. A multilingual embedding lets a model trained in one language serve many.

The engine does this blend natively — no application-side glue. On a predict, add a $semantic condition on the vector column: it votes the k nearest neighbours' labels, similarity-weighted, and fuses that with the ordinary token prediction (a weight in [0, 1] tunes the blend). $cluster instead quantizes the vector into a categorical code the model reasons over as an ordinary feature, and $knn/$nn do the same fusion with feature similarity. All three are documented in the Query Operator Reference. When the languages match the token model is already strong; when they don't, raise the $semantic weight so the embedding signal leads.

{ "from": "tickets", "predict": "intent",
  "where": { "embedding": { "$semantic": { "near": { "text": "reset my password" }, "k": 20, "weight": 0.5 } } } }

Notes

• Bring your own embeddings, unless the column declares an embedder. Aito stores and searches vectors; it does not encode text itself unless the column declares an embedder, in which case Aito calls that embedder for you. Use the same model for indexed rows and queries, and keep the dimensionality constant.

• Exact search. $nearest scans the (filtered) candidate set — no approximate index, no recall/latency knobs. Results are exact and reproducible. A large scan runs SIMD and in parallel: an unfiltered $nearest over 1M 384-dim vectors answers in about 65 ms on 8 threads.

  Available since: v2.11.5 for the parallel SIMD scan. From that build, $similarity scores can differ from earlier releases in their last digits (the summation order changed); the same build always returns the same scores.

• cosine vs dotProduct. cosine compares direction (magnitude ignored, vectors unit-normalised on ingest); dotProduct uses the raw inner product. A zero vector has no direction and is rejected on a cosine column.

• Nullable vectors. Mark the column nullable to allow rows without a vector; those rows are skipped by $nearest.

• Vectors are searched, not matched by value.

  Available since: v2.11.5

  A vector column is stored for similarity search only. A question about a vector's value returns a 400 that names the column: get on it, predict of it, and SQL count(DISTINCT v) or GROUP BY v. Use $nearest, $similarity or the hybrid blends instead.

  A vector column is also not used as exact-match evidence in predictions or relate. If your data repeats embeddings (for example an embedding of a category label), use the underlying column instead. To use a row's neighbours as evidence, add a $semantic condition.

Related

• Query Reference → Vector similarity
• Schema Design → Vector columns
• Inference (v2)