Vector Search & RAG (v2, Beta)

Aito v2 stores dense embeddings in a Vector column and ranks rows by similarity with the $nearest query operator. That gives you three things from one store:

  • Semantic retrieval β€” find the rows whose meaning is closest to a query (the retrieval half of RAG).
  • Hybrid search β€” combine the vector with BM25 full-text and a prediction ($p) in one query, so lexical, semantic, and contextual signals rank together.
  • Semantic inference β€” use a row's nearest neighbours as evidence to predict its label, which helps most exactly where token features fail (paraphrase, and especially across languages).

By default you bring your own vectors β€” compute them with your own model (any sentence/text encoder) and supply them as arrays of numbers; Aito stores, indexes, and searches them. Search is exact nearest-neighbour, so results are reproducible and there is no index to tune. (Optionally, a Vector column can declare an embedder so Aito embeds text for you β€” see below.)

The workflow

  1. Encode your text (or images, etc.) into vectors offline, with your model of choice. Keep the dimensionality fixed.
  2. Define a Vector column with that dimensions and a similarity (cosine for direction, dotProduct for raw inner product).
  3. Ingest each row with its vector as an array of numbers.
  4. Query with $nearest β€” optionally filtered, thresholded, and limited.

1–2. Schema

PUT /api/v2/schema/products
{ "type": "collection", "columns": {
  "id":        { "type": "Int" },
  "name":      { "type": "Text", "analyzer": "english" },
  "category":  { "type": "String" },
  "embedding": { "type": "Vector", "dimensions": 384, "similarity": "cosine" }
} }

cosine columns are unit-normalised on ingest, so $similarity is the cosine in [-1, 1]. See Schema Design β†’ Vector columns.

3. Ingest vectors

Vectors ride along with the rest of the row (batch or stream). Ingest in batches under the request-size limit for large corpora.

POST /api/v2/data/products/batch
[ { "id": 1, "name": "running shoe", "category": "shoes", "embedding": [0.02, -0.11, …] },
  { "id": 2, "name": "wool sock",    "category": "socks", "embedding": [0.07,  0.03, …] } ]

Optional: embed text automatically

Instead of bringing your own vectors, a Vector column can declare a pinned embedder and a from text column. Then you send text β€” at ingest and at query β€” and Aito calls the embedder to produce the vectors:

"embedding": { "type": "Vector", "dimensions": 384, "similarity": "cosine",
  "embedder": { "preset": "tei", "endpoint": "http://tei:8080/embed", "model": "intfloat/multilingual-e5-small" },
  "from": "name" }

Ingest rows with just the from text ({ "id": 1, "name": "running shoe" }) β€” Aito embeds name into embedding. Supplying the vector directly still works and skips the embedder. At query time, a text near/vector value is embedded with the same pinned model, so stored and query vectors share one space:

{ "where": { "$nearest": { "near": { "embedding": "cordless mouse" } } } }

preset is the endpoint wire format (openai / cohere / ollama / tei). The embedder is pinned to the column β€” changing the model means re-embedding.

4. Retrieve with $nearest

POST /api/v2/_query
{ "from": "products",
  "where": { "$nearest": {
    "near":  { "embedding": [0.01, -0.09, …] },
    "where": { "category": "shoes" },
    "having":{ "$similarity": { "$gte": 0.8 } },
    "limit": 10 } },
  "select": ["id", "name", "$similarity"] }

Rows come back ordered by $similarity descending. The nested where filters before scoring (so limit is the top-k among matches), having/threshold sets a minimum score, and limit caps the results. Full operator details: Query Reference β†’ Vector similarity.

RAG retrieval quality

For retrieval-augmented generation, $nearest is the retrieval step: embed the user's question, fetch the top-k passages, and hand them to your generator.

Because search is exact, retrieval quality is entirely a function of your embedding model β€” Aito reproduces it faithfully. On the public BEIR SciFact benchmark (5,183 abstracts, 300 claims) with all-MiniLM-L6-v2, $nearest scores nDCG@10 β‰ˆ 0.65, matching the published figure for that encoder. Swap in a stronger model and the retrieval numbers move with it.

Hybrid search: combine lexical, semantic, and predictive signals

Vector similarity is one signal, and it isn't always the right one. Which signal wins depends on the query:

  • BM25 full-text ($search/@@) nails exact tokens β€” names, SKUs, codes, rare terms β€” the things an embedding blurs into its neighbourhood.
  • Vector nails meaning β€” paraphrases, and answers that share no words with the query at all.
  • Prediction ($p) adds context β€” personalization, popularity, the probability of an outcome β€” which neither text nor vector can see.

They're complementary, not competing. On BEIR SciFact, bucketing queries by how much exact-term overlap they share with the answer, BM25 recall@10 runs from 0.00 (no shared word β€” it literally can't retrieve) to 0.96 (high overlap), while the vector arm is far flatter (0.53 β†’ 0.98): BM25 is an exact-match tool, the vector a semantic one. Over 300 queries, 76 were found only by the vector and 8 only by BM25 β€” each rescues cases the other misses. So the robust move is to combine them, which Aito does in one query.

Rank fusion β€” the hybrid query (scale-free default)

BM25 scores are unbounded; cosine is [-1, 1]. You can't just add them. The hybrid query sidesteps the scale mismatch with Reciprocal Rank Fusion β€” it fuses the two rankings, not the scores:

{ "from": "docs",
  "hybrid": {
    "match":   { "field": "content", "text": "wireless noise cancelling" },
    "nearest": { "embedding": "wireless noise cancelling" },
    "k": 60,
    "weights": { "match": 1.0, "nearest": 1.0 } },
  "select": ["title", "$score", "$bm25", "$similarity"], "limit": 10 }

It runs both arms, fuses over the union of their top hits (so a strong hit in either arm survives β€” the recall win), and returns $score (the fused rank score) plus each arm's raw $bm25/$similarity. weights down-weights a weaker arm; on a corpus where one signal dominates, equal-weight fusion can trail the stronger arm, and the weight is the fix.

Probabilistic blend β€” $multiply (composes with prediction)

Rank fusion can't compose with a $p. The alternative turns each signal into a probability lift and multiplies them in an orderBy:

{ "from": "products",
  "orderBy": { "$multiply": [
    { "$similarity": { "name": "wireless" } },
    { "$vectorIdf":  { "embedding": [0.01, -0.09, …] } } ] },
  "select": ["id", "name"], "limit": 10 }
  • $similarity is BM25 as a lift (exp(ΞΈΒ·bm25), already calibrated at ΞΈ=0.33).
  • $vectorIdf is the self-calibrating vector lift (1/p)^ΞΈ, where p is the cosine's rarity over the candidates β€” a "vector IDF", exactly BM25's (1/p)^0.33 form applied to similarity. The model-specific cosine scale is absorbed into p, so one ΞΈ (default 0.87) transfers across embedding models with no per-model tuning. ($vectorSimilarity is the raw-cosine lift and $vectorCosine the raw score, if you want those instead.)

The three-way blend β€” add prediction

Because $p is just another $multiply part, you can rank by P(outcome) Β· text-similarity Β· vector-similarity in a single query β€” a personalized semantic search. Rank product candidates (over a link) by purchase probability Γ— how well the name matches Γ— how close the meaning is:

{ "from": "impressions", "get": "product",
  "orderBy": { "$multiply": [
    { "$p":          { "$context": { "purchase": true } } },
    { "$similarity": { "name": "wireless" }, "theta": 0.33 },
    { "$vectorIdf":  { "embedding": [0.01, -0.09, …] }, "theta": 0.87 } ] },
  "select": ["name", "$p"], "limit": 10 }

This is where one store earns its keep: a product whose name shares no query word can still win β€” the vector matches its meaning and the shopper's history predicts a purchase β€” while a merely-lexical match without either sinks. Text, semantics, and prediction, resolved together in one orderBy.

Tuning β€” ranking vs calibration

Two knobs, and they're orthogonal:

  • Ranking depends only on the per-arm theta ratio ($similarity's ΞΈ vs $vectorIdf's ΞΈ) β€” a plain query parameter. A global exponent on the product is a ranking no-op. Tune the ratio (or lean on the calibrated defaults) to order results well.
  • Calibration (whether $score reads as a probability) depends on the overall scale. A pure ranking objective will happily choose over-confident values; if you need a calibrated P β€” to threshold, or to compose as an honest probability β€” fit a small logistic over the two signals offline (a once-per-corpus step; the query then just applies the weights).

Rule of thumb: hybrid (RRF) when you want a robust default with no tuning; $multiply when you want to fold in a prediction or need a composable score; $vectorIdf over $vectorSimilarity because it self-calibrates.

Semantic inference

Beyond retrieval, a row's nearest neighbours are strong evidence for its own label β€” a semantic k-NN classifier. This shines where surface form and meaning diverge and token features break down.

The sharpest case is cross-lingual. Trained on English and tested on other languages (Amazon MASSIVE intent classification, multilingual embeddings), a token classifier collapses toward the prior β€” the test words simply aren't in the training vocabulary β€” while nearest-neighbour voting in embedding space still lands the right intent:

English (in-language)German (cross-lingual)Spanish (cross-lingual)
token baseline0.790.130.15
semantic k-NN0.850.590.75

In-language, semantics edges a strong token model; cross-lingual, it wins 4–5Γ—. A multilingual embedding lets a model trained in one language serve many.

The engine does this blend natively β€” no application-side glue. On a predict, add a $semantic condition on the vector column: it votes the k nearest neighbours' labels, similarity-weighted, and fuses that with the ordinary token prediction (a weight in [0, 1] tunes the blend). $cluster instead quantizes the vector into a categorical code the model reasons over as an ordinary feature, and $knn/$nn do the same fusion with feature similarity. All three are documented in the Query Operator Reference. When the languages match the token model is already strong; when they don't, raise the $semantic weight so the embedding signal leads.

{ "from": "tickets", "predict": "intent",
  "where": { "embedding": { "$semantic": { "near": { "text": "reset my password" }, "k": 20, "weight": 0.5 } } } }

Notes

  • Bring your own embeddings. Aito stores and searches vectors; it does not encode text. Use the same model for indexed rows and queries, and keep the dimensionality constant.
  • Exact search. $nearest scans the (filtered) candidate set β€” no approximate index, no recall/latency knobs. Results are exact and reproducible.
  • cosine vs dotProduct. cosine compares direction (magnitude ignored, vectors unit-normalised on ingest); dotProduct uses the raw inner product. A zero vector has no direction and is rejected on a cosine column.
  • Nullable vectors. Mark the column nullable to allow rows without a vector; those rows are skipped by $nearest.