<!-- GENERATED FILE — do not edit. Source: core/core/src/main/scala/com/knowledgegarden/episto/core/query2/docs/V2QueryDocs.scala; regenerate with `./do v2-docs` (from core/). -->

v2 Query Operator Reference (Beta)

The complete operator vocabulary of the v2 (Query2) JSON query language — where conditions, orderBy, select, query types, and config. v2 is in beta; see the v2 Introduction for context and the v1 reference for the production API.

A query is one JSON object. The same from/where is evidence for a plain query, a prediction, a recommendation, or relation analysis — see the Inference guide.

Where conditions

where is an object of field → condition. A bare value means equality:

{ "from": "products", "where": { "category": "books" } }

Equality and tokens

Operator	Meaning	Example
(value) / $is	Equality (text fields tokenize)	{ "category": "books" }
$exact	Raw, non-featurized exact value	{ "color": { "$exact": "red" } }
$has	Token / membership. A single token, or an array (matches ANY)	{ "tags": { "$has": "sale" } } · { "tags": { "$has": ["sale","new"] } }
$has: { … }	Same-member conjunction on a member-link Set/Array (a set of foreign keys): matches a row that has ONE member whose linked row satisfies all the sub-conditions — not "some member is A and some other is B". $exists is an interchangeable spelling, and the same object form works on $refs reverse links	{ "items": { "$has": { "a": "fire", "b": "fighter" } } } — a basket with one item that is both

On a member-link's linked Text attribute (items.name where products.name is Text), $has matches by token — { "items.name": { "$has": "banana" } } finds baskets with a product named "pirkka banana" or "chiquita banana" (the shared token, not the whole phrase). select ["items.name"] still returns the whole names.

Text search

Text columns (type: "Text", with an analyzer) support three matchers:

Operator	Meaning	Example
$match	Tokenize the input and match rows containing all tokens (AND)	{ "description": { "$match": "wireless mouse" } }
$search	Full-text search — a Lucene-subset query language: bare terms (AND), "quoted phrases", OR, NOT / -term, and ( … ) groups	{ "description": { "$search": "(wireless OR bluetooth) mouse -refurbished" } }
$has	A single analyzed token	{ "description": { "$has": "wireless" } }

$match is the simple "all words present" matcher; $search is the expressive one when you need phrases, OR, or negation.

Ranges and prefixes

Operator	Meaning	Example
$gt $gte $lt $lte	Comparison — numeric columns compare numerically; String columns compare lexicographically (the date-as-string cutoff idiom)	{ "n": { "$gt": 20 } } · { "month": { "$lte": "2024-07" } }
$startsWith	Prefix match on a String column	{ "sku": { "$startsWith": "FI-" } }
$mod	Modulo (numeric) — [divisor, remainder] or {divisor, remainder}	{ "n": { "$mod": [3, 1] } }
$numeric	Adaptive neighbourhood (binned), not an exact match	{ "n": { "$numeric": 101 } }

$numeric matches a neighbourhood: Aito bins the column adaptively, so $numeric: 101 draws on nearby values instead of overfitting the single exact row — the mechanism that makes a number useful as evidence. A $gt on a non-numeric column fails loud.

Date components

Date columns are stored ISO-8601 and queried by component:

Operator	Meaning
$year $month $day	Calendar components
$weekday / $dayOfWeek	Day of week
$dayOfYear	1–366
$match	Match the date as text

{ "from": "orders", "where": { "created": { "$weekday": 6 } } }

Nullability

Operator	Meaning	Example
$exists / $defined	Has a value (true) or is null/absent (false)	{ "color": { "$exists": true } }
null equality	{ "field": null } is an alias for { "field": { "$exists": false } }	{ "color": null }

Logical combinators

Operator	Meaning	Example
$and $or $not	Combine nested where-objects	{ "$or": [ { "color": "red" }, { "size": "M" } ] }
field-scoped $or / $in	Is-any-of: a value list on one field	{ "id": { "$or": ["a", "b", "c"] } } · { "id": { "$in": [1, 2] } }
field-scoped $not	Negated equality on one field	{ "status": { "$not": "done" } }
field-scoped $and	Combine field-scoped conditions (e.g. exclude a basket)	{ "id": { "$and": [ { "$not": "a" }, { "$not": "b" } ] } }
$on	Observe a condition only where another holds ([prop, on])	{ "$on": [ { "color": "red" }, { "size": "M" } ] }
$group	A correlated theme that votes by mutual corroboration (compression), not OR — for predictions where a set of tokens should count as one decorrelated signal	{ "$group": [ { "tag": "a" }, { "tag": "b" } ] }

Combinators work on both collection tables and legacy table worlds.

Vector similarity

$nearest ranks rows by similarity to a query vector on a Vector column. It is a single where proposition that produces its own ordered, scored result set — the nearest rows first, each carrying its score as $similarity.

{ "from": "products",
  "where": { "$nearest": {
    "near":  { "embedding": [0.12, -0.03, 0.88] },
    "where": { "category": "shoes" },
    "having":{ "$similarity": { "$gte": 0.8 } },
    "limit": 10 } },
  "select": ["id", "name", "$similarity"] }

Key	Meaning
near	{ <vector-column>: [query vector] } — the column to score and the query vector. Its length must equal the column's dimensions; a mismatch (or a non-finite component) is rejected.
where	Optional candidate filter — any where-object. Scoring runs only over the survivors, so limit is the top-k among matches, never a globally-nearest row the filter should have excluded.
having	Optional score floor on $similarity: { "$similarity": { "$gte": t } } (inclusive) or { "$gt": t } (exclusive).
threshold	Sugar for having: { "$similarity": { "$gte": t } }.
limit	Top-k cap — return at most this many nearest rows.

$similarity is the per-row score, exposed as a selectable field; rows come back ordered by it descending (an implicit orderBy $similarity). Its meaning follows the column's similarity setting: for cosine it is the cosine similarity in [-1, 1] (vectors are unit-normalised on ingest, so scoring is an exact dot product); for dotProduct it is the raw inner product. Rows whose vector is null are skipped, not scored.

$nearest is exact nearest-neighbour (a full scan of the candidates), not an approximate index — results are exact and reproducible, with ties broken by row order.

Vectors as prediction evidence — $cluster / $semantic

Beyond search, a Vector column can drive prediction. On a predict query, two operators turn similarity into evidence.

$cluster quantizes a vector into a categorical code (k-means), so it becomes an ordinary discrete feature the model reasons over:

{ "from": "tickets", "predict": "category",
  "where": { "embedding": { "$cluster": { "vector": [0.1, 0.9, 0.2], "k": 64 } } } }

Key	Meaning
vector	The query vector to code. Or text — embedded via the column's embedder (see Vector search).
k	Codebook size (number of clusters).
as	Optional name for the derived code column (default <column>_cluster).

The vector is coded against the column's codebook and the condition rewrites to a plain equality on the code column — so the vector becomes ordinary categorical evidence. A single full-vector cluster recovers most of the exact-kNN lift.

$semantic blends the k nearest neighbours' predict-field values into the prediction — semantic-kNN late fusion, the continuous-similarity form:

{ "from": "tickets", "predict": "category",
  "where": { "embedding": { "$semantic": { "near": [0.1, 0.9, 0.2], "k": 20, "weight": 0.5 } } } }

Key	Meaning
near	The query vector, or text (embedded via the column's embedder).
k	Neighbours to vote (default 10).
weight	Blend weight in [0, 1]: 0 = the token/naive-Bayes baseline only, 1 = pure kNN vote, 0.5 (default) = an even blend.

The k nearest rows vote their predict field, similarity-weighted, and the vote is fused with the ordinary prediction. $knn/$nn do the same with feature similarity instead of vector similarity (see below).

Text introspection

On a text field you can also read its analysis: $size (row count), $tokenCount (distinct tokens), $distinctCount (distinct values). Select them with a field.$op token — e.g. "select": ["note.$tokenCount"]. The value is a property of the whole column, so every returned row carries the same number (a per-row constant); pair it with "limit": 1 to read it once.

Inverse links — $refs

A dotted path follows a forward link — from a row to the row it points at (order.buyer resolves the order's user, and buyer.name reads that user's name). $refs follows a link the other way: from a row to the rows that point at it. $refs.<table>.<fk> is the set of <table> rows whose link <fk> targets this row — e.g. on a user, $refs.orders.buyer is that user's orders (the orders whose buyer is this user). Append an attribute to project it over that set: $refs.orders.buyer.product is the products of those orders.

It is an ordinary field — you query, select, and predict over it with the same operators a stored member-set uses. The query is identical whether the relation is stored on the row or discovered via the back-link; nothing is copied onto the referring row.

Link resolution is lazy, and a link's cross-table enrichment is optional. A link's target is resolved at query time, not validated when the schema is created — so you may declare a link before its target table exists, and a link whose target is missing, renamed, or deleted does not raise an error. Its linked columns and its cross-table prediction signal are simply absent for that query, and the result degrades gracefully to what the row itself carries. (This is the long-standing rep1 behaviour, preserved by design.) Cross-table priors additionally require the target to be a rep2 collection; a link to a rep1 (type: "table") target resolves as absent the same way. So if a predict seems to ignore a linked signal, confirm the target table exists and is a rep2 collection.

Where — over the referring set:

Condition	Meaning	Example
$exists	Has ≥1 referring row	{ "$refs.orders.buyer": { "$exists": true } } — users with an order
$has on a projected attribute	A referring row whose attribute matches	{ "$refs.orders.buyer.product": { "$has": "gold" } } — users with a "gold" order
$exists/$has with an object (same-member filter)	Restrict the referring rows before the existence check — ALL conditions hold on the SAME referring row; a condition col may itself be a forward link (item.tier — one more hop)	{ "$refs.orders.buyer": { "$exists": { "item.tier": "premium" } } }
<link>.$refs.… (forward-then-reverse)	Resolve this row's forward link first, then its inverse links — two hops	{ "sender.$refs.edges.subject": { "$exists": true } } — emails whose sender has an edge

Row-level $and of two $refs conditions intersects, e.g. a user with both a gold order and a web order:

{ "from": "users",
  "where": { "$and": [
    { "$refs.orders.buyer.product": { "$has": "gold" } },
    { "$refs.orders.buyer.channel": { "$has": "web" } } ] } }

Select — project the attribute over each row's referring set (an array per row):

{ "from": "users", "select": ["name", { "products": "$refs.orders.buyer.product" }] }
// → { "name": "Alice", "products": ["gold", "silver"] }, …

Predict — use a $refs condition as where evidence to predict a field from the back-linked rows, without materialising anything onto the row:

{ "from": "users",
  "where": { "$refs.orders.buyer.product": { "$has": "gold" } },
  "predict": "segment" }

A same-member conjunction — one referring row that matches several attributes at once (an order that is gold and on web) — is the object form: { "$refs.orders.buyer": { "$exists": { "product": "gold", "channel": "web" } } }. Contrast the row-level $and above, which matches any gold order and any (possibly different) web order.

Ordering — orderBy

Form	Meaning
"price" / { "field": "price", "desc": true }	Sort by a field
{ "$asc": "price" } / { "$desc": "price" }	Direction shorthands for the same (also work on "$p"/"$f"/"$lift", e.g. { "$asc": "$f" })
"$p"	Rank by prediction probability
"$f"	Rank get candidates by their frequency (count under the where)
"$lift"	Rank get candidates by lift (how much the where raises each value's probability)
{ "$p": { "$context": <where> } }	Rank candidates by P(condition in the from/context table)
{ "$similarity": { <textField>: <text> } }	Rank rows by the BM25 relevance of the field against the text (the search scorer — real scores, not an approximation). On a get over a link, the field may be a linked text column: candidates score by their link target's BM25. $sameness is an accepted alias.
{ "$multiply": [ … ] }	Rank by the per-row product of parts: the string "$similarity" (references the query's $match where-term), a { "$similarity": {field: text} } object, or a { "$p": { "$context": … } } part

{ "from": "impressions", "get": "product",
  "orderBy": { "$p": { "$context": { "click": true } } },
  "select": ["$value", "$p"] }

Projection — select

Plain column names, computed value-expression columns (arithmetic over fields — see Derived fields → let), plus the predictive columns below. On a get over a link, the link TARGET's columns are selectable too — each candidate resolves them on its target row (get: "product" + select: ["name", "price"] returns each candidate product's own attributes):

Column	Meaning
$value	The candidate value
$p	Its predicted probability
$score	The value used by orderBy — for a predict/recommend ranking, equal to $p
$f	The candidate's frequency — count of rows where the predicted field equals this value, under the where
$lift	The candidate's lift — how much the where evidence raises its probability over its base rate
$why	The explanation factor tree (see the Inference guide). The parameterized form { "$why": { "highlight": { "posPreTag": "<b>", "posPostTag": "</b>" } } } additionally marks the matched evidence tokens in the explanation with your tags
$highlight	On a search query: the matched field value with the query's tokens marked with tags — the search-hit form of $why's highlight
$matches	On a search query: the matched tokens and their positions in the field value
{ alias: "source" }	Rename a column in the output — { "demandScore": "$score" }
{ alias: <value-expression> }	A computed column — { "margin": { "$subtract": ["revenue", "cost"] } }. Any let value-expression (arithmetic over fields/constants; see Derived fields); evaluated the same way as let, so select and let share one vocabulary
{ alias: { "$p": { "$context": <where> } } }	A named per-candidate contextual probability column
{ "$sum" | "$mean": { "$context": <field> } }	Per-candidate aggregate of a context-table field over the candidate's where-filtered rows (bare, or wrapped in an alias) — e.g. a per-product conversion rate { "$mean": { "$context": "purchase" } }
$count / $sum:f / $avg:f / $min:f / $max:f	Relational aggregate over the whole where selection — collapses all matching rows to ONE hit (SQL count(*), sum(f), …), ignoring offset/limit. Skips null cells; null over zero rows. Distinct from the per-candidate { "$sum": { "$context": … } } above; mixing an aggregate with a plain field is rejected (SQL would need a GROUP BY)

Selecting an unsupported computed column fails loud (a 4xx), never a silent empty value. Aggregates and $f require candidates whose rows live in the from table itself — a dotted-path get (e.g. get: "context.week") is a known limitation (returns zeros; tracked on the gap matrix).

Derived fields — let

let defines inline computed fields from value-expressions, usable in select and orderBy. It is the join-free way to compute derived columns — margin, ratios, and the like — without a CTE or self-join:

{ "from": "sales",
  "let": { "margin":    { "$subtract": ["revenue", "cost"] },
           "marginPct": { "$divide": [ { "$subtract": ["revenue", "cost"] }, "revenue" ] } },
  "select": ["id", "margin", "marginPct"], "orderBy": "margin" }

A value-expression is one of:

Form	Meaning
"revenue" / { "$field": "revenue" }	A field's value
123 / { "$const": 2.0 }	A constant
{ "$sum": [a, b, …] }	Sum of the parts
{ "$subtract": [a, b] }	a − b
{ "$multiply": [a, b, …] }	Product of the parts
{ "$divide": [a, b] }	a / b
{ "$pow": [base, exp] }	base raised to exp (element-wise)
{ "$normalize": expr }	Scale so the column's values sum to 1.0 across the queried rows (a cross-row transform — e.g. a revenue share)

Parts nest arbitrarily. A missing or non-numeric part, or division by zero, makes that row's value undefined.

from — collections, views, and inline relations (Beta)

from names what a query reads. Besides a plain collection it accepts a view (a derived relation saved in the schema) or an inline relation expression (an ad-hoc join or union for a single query). All are queried, filtered, projected, and predicted exactly like a collection — a view is just the saved, materialised form of the same relation operators.

from form	Example	Reads
collection	"orders"	one collection
view	"search_index"	a saved view — union-merge or link-join
inline link-join	{ "from": "orders", "join": { "table": "products", "on": { "$=": ["product", "id"] } } }	a base foreign-key column exposed as a navigable link for this query; joined columns navigate via dotted paths (product.name), no content copied
inline union	{ "union": ["appLogs", "backendLogs"] }	several collections stacked into one relation (heterogeneous schemas unify column-by-column; lazy, no copy)
projected union	{ "union": [ { "from": "customers", "select": { "id": "id", "content": { "$text": ["name", "notes"] } } }, … ] }	each source mapped into one common schema — "col" renames a column, { "$const": v } a literal, { "$multiply": [...] } an arithmetic column (same vocabulary as let), and { "$text": [...] } concatenates Text fields into one analyzer-backed content column so a single $match/$search spans every source

Views are created and refreshed in the schema — see Schema Design → Views for the full treatment (both kinds, worked examples, _refresh semantics), and Relationships, Links & Joins for how link-joins behave in a query. A view is a materialised snapshot — rebuild it with POST /api/v2/schema/{view}/_refresh after its sources change; an inline $text union is tokenised per query, so for a hot cross-collection search prefer a persisted union view (built once).

Query types (top level)

Field	Meaning
predict: "field"	Rank a field's values by probability given where
recommend: "linkField" + goal: {…}	Rank link candidates by P(goal)
relate: ["field", …]	Return statistical relations (lift, info-gain), not rows
search: { "field": …, "text": … }	Sugar: text match + rank by relevance
get: "field"	Open a field's distinct values as candidates
basedOn: ["field", …]	Project linked-table fields in as cross-table evidence
predict: "field.$feature"	Non-exclusive (multi-label) predict of a member/array field: each member value is scored independently by P(field has value | where) — for tags, baskets, categories. The prediction target determines the mode: predict X is exclusive (candidates are distinct whole values), predict X.$feature is per-member. The v1-style exclusiveness flag is deprecated — if given, it must agree with the target or the request is rejected

Endpoints: /api/v2/_query is the unified entry; /_predict, /_search, /_recommend, /_relate are aliases that take the same body; /_batch runs a JSON array of queries. Also on v2, engine-dispatched (collections run natively; rep1 tables reuse the v1 pipeline):

Endpoint	Meaning
/_evaluate	Train/test-split quality measurement — see the Evaluation guide. Supports test conditions, { "$index": n } row selection, sampling selectors ($sample — incl. the {n, of, seed} form — and $hash with $mod, in test and testSource.where), testSource {from, where, limit}, $get bindings, operator conditions (e.g. a $lte month cutoff), and per-query config (inference profile + train-fitted calibration)
/_estimate	Estimate of a numeric field given the where. Default: posterior-mean (probability-weighted mean over the predicted candidates, weightedAverage why). model: "regression" (pure transformation-pipeline regression) and model: "knn" (K-NN adjusted) run the v1 estimation algorithms; both require plain-equality where conditions
/_aggregate	$sum / $mean (+ variance stats) / $min / $max / $f over the where-matching rows — array form (["price.$mean"], keys = the spec strings) or aliased object form ({ "conversion": "purchase.$mean" })
/_match	A field's distinct values ranked by frequency under the where (lowered to get + orderBy: "$f")

Config

Field	Meaning
config.ai	Inference profile preset — v2/high (And + Group re-expression, the default), group (Group only), v1/and (legacy And), or fast/flat (plain naive Bayes); unknown names fail loud
config.calibrate	Temperature-calibrate probabilities once per DB state

Feature-based nearest rows — $knn / $nn

Rank rows by feature similarity to an exemplar (text fields score by BM25). This is a where operator (the feature analog of the vector $nearest / $semantic above), not a config setting:

{ "from": "msgs", "where": { "$knn": { "near": { "text": "sim card" } } }, "limit": 5 }
{ "from": "msgs", "where": { "$nn": { "near": { "text": "sim card" }, "threshold": 0.5 } } }

$nn keeps only rows scoring above threshold (default 0.0). The v1 array form { "$nn": [ { "text": "…" } ] } is accepted with a single exemplar. $similarity is selectable on the results.

On a predict query, $knn/$nn become feature-kNN classification — the feature analog of $semantic: the k feature-nearest neighbours vote the predict field, blended with the ordinary prediction.

{ "from": "tickets", "predict": "agent",
  "where": { "$knn": { "near": { "text": "password reset" }, "k": 2, "weight": 0.5 } } }

k (neighbours to vote, default 10) and weight (blend in [0, 1], default 0.5) apply only in this predict context; as a plain search operator $knn/$nn page by the query's limit.

Population restriction — nested from

from accepts a restriction object instead of a table name:

{ "from": { "from": "invoices", "where": { "customer_id": "CUST-0000" }, "limit": 50000 },
  "where": { "gl_code": "1600" },
  "predict": "approver" }

Rows failing the inner where are masked out of the examined table — the population the query sees. This is different from adding the filter to the outer where: where is evidence (it conditions the prediction and contributes lift), while the nested from restricts the row population, so base rates, support counts and candidates all compute over the subset (the multi-tenancy idiom). The optional inner limit caps the restricted population to its first N rows. One restriction level; a restriction matching no rows, deeper nesting, and unsupported combinations (relate, bare search, $nearest/$knn, config.calibrate) all fail loud.

Itemset mining — relate: {"$patterns": …}

The market-basket primitive: which sets of a field's values co-occur more than chance. Candidates are the fields' MEMBERS — array/set elements, analyzed text tokens, or scalar values — mined greedily into variable-size itemsets (pairs seed, grow while the MDL gain holds, up to 6 items, support ≥ 2), then each itemset is scored against the where condition:

{ "from": { "from": "invoices", "where": { "customer_id": "CUST-0000" } },
  "where": { "gl_code": "1600" },
  "relate": { "$patterns": ["vendor", "category", "amount_band"] } }

Each hit carries related (the $and itemset), condition, lift, info, f (the itemset's support count) and n. Without a where, itemsets rank by their own support. The {"$related": {"relate": [...], "k": n, "to": {…}}} wrapper bounds the candidate set (k, default 32) and adds a narrowing target. Nested from composes (the multi-tenancy shape above): support counts and n are computed over the restricted population.

Not yet supported

• Nested from with plain relate — typed 501 (bit-universe alignment); relate $patterns DOES compose with nested from.
• basedOn $patterns (conjunction mining as predict evidence) — fails loud; planned.
• Arithmetic score composition beyond $multiply — v1's $divide, $subtract, $pow, $normalize and friends are not ported; only $multiply composes ranking values on v2 (let covers derived fields).

All of these fail loud rather than degrade — a 4xx/501 with the reason, never a silently wrong 200.

Related

• Inference (v2) · Schema Design (v2) · CollectionDb (v2)