Vector Search & RAG (v2, Beta)

Aito v2 stores dense embeddings in a Vector column and ranks rows by similarity with the $nearest query operator. That gives you two things from one store:

• Semantic retrieval — find the rows whose meaning is closest to a query (the retrieval half of RAG).
• Semantic inference — use a row's nearest neighbours as evidence to predict its label, which helps most exactly where token features fail (paraphrase, and especially across languages).

By default you bring your own vectors — compute them with your own model (any sentence/text encoder) and supply them as arrays of numbers; Aito stores, indexes, and searches them. Search is exact nearest-neighbour, so results are reproducible and there is no index to tune. (Optionally, a Vector column can declare an embedder so Aito embeds text for you — see below.)

The workflow

1. Encode your text (or images, etc.) into vectors offline, with your model of choice. Keep the dimensionality fixed.
2. Define a Vector column with that dimensions and a similarity (cosine for direction, dotProduct for raw inner product).
3. Ingest each row with its vector as an array of numbers.
4. Query with $nearest — optionally filtered, thresholded, and limited.

1–2. Schema

PUT /api/v2/schema/products
{ "type": "collection", "columns": {
  "id":        { "type": "Int" },
  "name":      { "type": "Text", "analyzer": "english" },
  "category":  { "type": "String" },
  "embedding": { "type": "Vector", "dimensions": 384, "similarity": "cosine" }
} }

cosine columns are unit-normalised on ingest, so $similarity is the cosine in [-1, 1]. See Schema Design → Vector columns.

3. Ingest vectors

Vectors ride along with the rest of the row (batch or stream). Ingest in batches under the request-size limit for large corpora.

POST /api/v2/data/products/batch
[ { "id": 1, "name": "running shoe", "category": "shoes", "embedding": [0.02, -0.11, …] },
  { "id": 2, "name": "wool sock",    "category": "socks", "embedding": [0.07,  0.03, …] } ]

Optional: embed text automatically

Instead of bringing your own vectors, a Vector column can declare a pinned embedder and a from text column. Then you send text — at ingest and at query — and Aito calls the embedder to produce the vectors:

"embedding": { "type": "Vector", "dimensions": 384, "similarity": "cosine",
  "embedder": { "preset": "tei", "endpoint": "http://tei:8080/embed", "model": "intfloat/multilingual-e5-small" },
  "from": "name" }

Ingest rows with just the from text ({ "id": 1, "name": "running shoe" }) — Aito embeds name into embedding. Supplying the vector directly still works and skips the embedder. At query time, a text near/vector value is embedded with the same pinned model, so stored and query vectors share one space:

{ "where": { "$nearest": { "near": { "embedding": "cordless mouse" } } } }

preset is the endpoint wire format (openai / cohere / ollama / tei). The embedder is pinned to the column — changing the model means re-embedding.

4. Retrieve with $nearest

POST /api/v2/_query
{ "from": "products",
  "where": { "$nearest": {
    "near":  { "embedding": [0.01, -0.09, …] },
    "where": { "category": "shoes" },
    "having":{ "$similarity": { "$gte": 0.8 } },
    "limit": 10 } },
  "select": ["id", "name", "$similarity"] }

Rows come back ordered by $similarity descending. The nested where filters before scoring (so limit is the top-k among matches), having/threshold sets a minimum score, and limit caps the results. Full operator details: Query Reference → Vector similarity.

RAG retrieval quality

For retrieval-augmented generation, $nearest is the retrieval step: embed the user's question, fetch the top-k passages, and hand them to your generator.

Because search is exact, retrieval quality is entirely a function of your embedding model — Aito reproduces it faithfully. On the public BEIR SciFact benchmark (5,183 abstracts, 300 claims) with all-MiniLM-L6-v2, $nearest scores nDCG@10 ≈ 0.65, matching the published figure for that encoder. Swap in a stronger model and the retrieval numbers move with it.

Semantic inference

Beyond retrieval, a row's nearest neighbours are strong evidence for its own label — a semantic k-NN classifier. This shines where surface form and meaning diverge and token features break down.

The sharpest case is cross-lingual. Trained on English and tested on other languages (Amazon MASSIVE intent classification, multilingual embeddings), a token classifier collapses toward the prior — the test words simply aren't in the training vocabulary — while nearest-neighbour voting in embedding space still lands the right intent:

	English (in-language)	German (cross-lingual)	Spanish (cross-lingual)
token baseline	0.79	0.13	0.15
semantic k-NN	0.85	0.59	0.75

In-language, semantics edges a strong token model; cross-lingual, it wins 4–5×. A multilingual embedding lets a model trained in one language serve many.

The engine does this blend natively — no application-side glue. On a predict, add a $semantic condition on the vector column: it votes the k nearest neighbours' labels, similarity-weighted, and fuses that with the ordinary token prediction (a weight in [0, 1] tunes the blend). $cluster instead quantizes the vector into a categorical code the model reasons over as an ordinary feature, and $knn/$nn do the same fusion with feature similarity. All three are documented in the Query Operator Reference. When the languages match the token model is already strong; when they don't, raise the $semantic weight so the embedding signal leads.

{ "from": "tickets", "predict": "intent",
  "where": { "embedding": { "$semantic": { "near": { "text": "reset my password" }, "k": 20, "weight": 0.5 } } } }

Notes

• Bring your own embeddings. Aito stores and searches vectors; it does not encode text. Use the same model for indexed rows and queries, and keep the dimensionality constant.
• Exact search. $nearest scans the (filtered) candidate set — no approximate index, no recall/latency knobs. Results are exact and reproducible.
• cosine vs dotProduct. cosine compares direction (magnitude ignored, vectors unit-normalised on ingest); dotProduct uses the raw inner product. A zero vector has no direction and is rejected on a cosine column.
• Nullable vectors. Mark the column nullable to allow rows without a vector; those rows are skipped by $nearest.

Related

• Query Reference → Vector similarity
• Schema Design → Vector columns
• Inference (v2)