Availability: API v1

Any schema which is a valid Aito table schema.

Table schema describes the structure of the table in a formal language. The schema describes all fields (or columns), data types of the fields, and information to help Aito preprocess your data. For example what language a textual data contains.

The contents of the schema depends on the data that will be inserted into the database.

{
  "type": string,
  "columns": object
}

Availability: API v1 · API v2 (beta)

Any object which is valid according to the database schema.

The contents of the object depends on the data inserted into the database. If for example you have a products table which has fields name and price, your object could look like:

{ "name": "My product", "price": 172.19 }

Availability: API v1

Boolean column type.

When column is a boolean, the only accepted values are true and false.

{
  "type": string,
  "nullable": boolean,
  "link": string
}

Availability: API v1

Double-precision floating-point number.

{
  "type": string,
  "nullable": boolean,
  "link": string
}

Availability: API v1

Integer column type.

{
  "type": string,
  "nullable": boolean,
  "link": string
}

Availability: API v1

String column type.

The string data type is a primitive version of the Text type. The value is turned into a single feature. For example "lazy black cat" becomes 1 feature: "lazy black cat".

{
  "type": string,
  "nullable": boolean,
  "link": string
}

Availability: API v1

Text column type.

The text data type enables smart textual analysis of strings. A text column has an analyzer which defines how the text can be split into words or tokens, which are used as features during inference.

{
  "type": string,
  "analyzer": Analyzer,
  "nullable": boolean,
  "link": string
}

Availability: API v1

Json column type.

The json datatype type can have an arbitrary json value The value is turned into a single feature. For example {"a":[1, 2, 3], "b":true} becomes 1 feature: {"a":[1, 2, 3], "b":true}.

{
  "type": string,
  "nullable": boolean,
  "link": string
}

Availability: API v1

Array column type.

The array data type allows storing lists of values of the same type. Each element in the array becomes a separate feature, enabling queries that match any element in the array.

Syntax

Array types are specified by appending [] to the base type name:

• Int[] - Array of 32-bit integers
• Long[] - Array of 64-bit integers
• Decimal[] - Array of decimal numbers
• String[] - Array of strings
• Boolean[] - Array of booleans

Schema Example

{
  "schema": {
    "products": {
      "type": "table",
      "columns": {
        "id": { "type": "Int" },
        "name": { "type": "String" },
        "tags": { "type": "String[]" },
        "prices": { "type": "Decimal[]" },
        "ratings": { "type": "Int[]" }
      }
    }
  }
}

Data Example

{
  "id": 1,
  "name": "Laptop",
  "tags": ["electronics", "computer", "portable"],
  "prices": [999.99, 899.99, 849.99],
  "ratings": [5, 4, 5, 4, 3]
}

Querying Arrays

Direct Array Match

When querying array fields, you can provide an array of values to match. This matches rows where the array field contains all the specified values:

{
  "from": "products",
  "where": {
    "tags": ["electronics", "computer"]
  }
}

This query matches products where the tags array contains both "electronics" AND "computer".

Using $has Operator

The $has operator checks if an array contains a specific value:

{
  "from": "products",
  "where": {
    "tags": { "$has": "electronics" }
  }
}

This matches products where the tags array contains "electronics".

Using $match Operator

The $match operator matches arrays containing any of the specified values:

{
  "from": "products",
  "where": {
    "tags": { "$match": ["electronics", "clothing"] }
  }
}

This matches products where tags contains "electronics" OR "clothing".

Numeric Array Comparisons

For numeric arrays, comparison operators work element-wise. The condition matches if any element in the array satisfies the comparison:

{
  "from": "products",
  "where": {
    "prices": { "$lt": 900 }
  }
}

This matches products where any price in the array is less than 900.

You can also use arrays with comparison operators:

{
  "from": "products",
  "where": {
    "ratings": { "$gte": [4, 5] }
  }
}

Use Cases

• Tags and categories: Store multiple labels per item
• Multi-valued attributes: Store multiple values for a single field
• Historical data: Store a series of measurements or values
• Recommendations: Store lists of related items or preferences

Notes

• Arrays can be nullable: use "nullable": true in the column definition
• Empty arrays ([]) are valid values
• Array elements must all be of the same type
• When querying with an array value directly (e.g., "tags": ["a", "b"]), all values must match
• The $has operator checks for a single value
• The $match operator checks for any of multiple values

{
  "type": string,
  "nullable": boolean
}

Availability: API v1

Aito has several built-in analyzers and they are selected by using their name in the "analyzer" field of a text column. For instance:

{ "analyzer": "english" }

The built-in analyzers include:

• Standard Analyzer:
• Name: "standard"
• A good default analyzer which Works well in most languages. The analyzer generates features based on the Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29. The standard analyzer filters English stop words that are normally not useful.
• E.g: "the cats are running" will be break down into "cats", "running".
• Whitespace Analyzer:
• Name: "whitespace"
• The analyzer breaks the text into features whenever it encounters a whitespace character. Adjacent sequences of non-Whitespace characters form tokens.
• E.g: "the cats are running" will be break down into "the", "cats", "are", and "running".
• Language Analyzer:
• Alias: the language name or the language ISO 639-1 Code (except some special case)
• A Language Analyzer with the default setting (no stop words or keywords).
• See Language Analyzer for supported languages and its aliases.

Availability: API v1

The Character N-gram Analyzer breaks text into n-gram features.

For example, the following n-gram analyzer:

{ "type": "char-ngram", "minGram": 3, "maxGram": 3 }

would break the text "the cats are running" into the following list of features:

["the", "he ", "e c", " ca", "cat", "ats", "ts ", "s a", " ar", "are", "re ", "e r", " ru", "run", "unn", "nni", "nin", "ing"]

The analyzer can be useful for languages that don’t use spaces or that have long compound words, like German.

{
  "type": string,
  "minGram": integer,
  "maxGram": integer
}

Availability: API v1

The Delimiter Analyzer breaks text into features whenever encounters a specified delimiter character.

With the trimWhitespace option, the analyzer trims the whitespace surrounding a feature.

For example, the following analyzer:

{
  "type": "delimiter",
  "delimiter": ",",
  "trimWhitespace": true
}

would break the text "the, cats,are, running" into 4 features:

["the", "cats", "are", "running"]

{
  "type": string,
  "delimiter": string,
  "trimWhitespace": boolean
}

Availability: API v1

Language Analyzers aim to analyze text of a specific language.

When using a language analyzer, text is analyzed into lower-case word stem features. For example, using the following english analyzer:

{ "type": "language", "language": "english" }

a text "the cats are running" will be broken into 4 word stem features:

["the", "cat", "ar", "run"]

The value of the "language" parameter specifies which language will be used. The value can be the name or the ISO 639-1 code of the language. The full list is shown as below:

Language	Name	ISO code
Arabic	arabic	ar
Armenian	armenian	hy
Basque	basque	eu
Brazilian Portuguese	brazilian	pt-br
Bulgarian	bulgarian	bg
Catalan	catalan	ca
Chinese, Japanese, Korean	cjk	cjk
Czech	czech	cs
Danish	danish	da
Dutch	dutch	nl
English	english	en
Finnish	finnish	fi
French	french	fr
Galician	galician	gl
German	german	de
Greek	greek	el
Hindi	hindi	hi
Hungarian	hungarian	hu
Indonesian	indonesian	id
Irish	irish	ga
Italian	italian	it
Latvian	latvian	lv
Norwegian	norwegian	no
Persian	persian	fa
Portuguese	portuguese	pt
Romanian	romanian	ro
Russian	russian	ru
Spanish	spanish	es
Swedish	swedish	sv
Thai	thai	th
Turkish	turkish	tr

The language analyzers support filtering the stop words (common words that are normally not useful). Each language has a list of default stop words for filtering that can be enabled through the useDefaultStopWords" parameter. Some common English stop words are:

  "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", 
  "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", 
  "such", "that", "the", "their", "then", "there", "these", 
  "they", "this", "to", "was", "will", "with"

By default, "useDefaultStopWords" is set as false. The following analyzer:

{
  "type": "language",
  "language": "english",
  "useDefaultStopWords": true
}

would break the text "the cats are running" into 2 features:

["cat", "run"]

It is also possible to specify a set of words that would be filtered through the "customStopWords" parameter and a set of words that would not be analyzed through the "customKeyWords" parameter. The following analyzer:

{
  "type": "language",
  "language": "english",
  "useDefaultStopWords": false,
  "customStopWords": ["cats"],
  "customKeyWords": ["running"]
}

would break the text "the cats are running" into 3 features:

["the", "ar", "running"]

{
  "type": string,
  "language": string,
  "useDefaultStopWords": boolean,
  "customStopWords": [string],
  "customKeyWords": [string]
}

Availability: API v1

The Token N-gram Analyzer breaks text into token n-grams (shingles) based on a source analyzer. In other words, it combines the features of the source analyzer into new features.

For example, the following Token N-gram Analyzer:

{
  "type": "token-ngram",
  "source": "english",
  "minGram": 1,
  "maxGram": 2,
  "tokenSeparator": "_"
}

would breaks the text "the cat is running" into the following list of features:

["the", "the_cat", "cat", "cat_ar", "ar", "ar_run", "run"]

{
  "type": string,
  "source": Analyzer,
  "minGram": integer,
  "maxGram": integer,
  "tokenSeparator": string
}

Availability: API v1 · API v2 (beta) — v2 form: $match

Operator to check if a textual field fuzzy matches a given string.

Case insensitive. The matched text is split to tokens with the analyzer specified for the field in schema. For example { "$match": "great programmers" } will match strings "Bob is the greatest programmer!", and "Programmers are having great fun" if the field is properly analyzed with the English analyzer.

{
  "$match": $toString or GetValueExpression or string
}

Availability: API v1 · API v2 (beta) — v2 form: $startsWith

Operator to check if a textual field starts with a given string. Case sensitive.

{
  "$startsWith": $toString or GetValueExpression or string
}

Availability: API v1 · API v2 (beta) — v2 form: $gt

Operator to check if a field is greater than a given value.

Works with numeric types (Int, Decimal) and String fields (lexicographic comparison).

Syntax:

{ "fieldName": { "$gt": value } }

In a where clause:

{
  "from": "products",
  "where": {
    "price": { "$gt": 10.00 }
  }
}

This returns all products with a price greater than 10.00.

{
  "$gt": GetValueExpression or integer or integer or number or null or ArrayValue or $toString or boolean or string or Json
}

Availability: API v1 · API v2 (beta) — v2 form: $gte

Operator to check if a field is greater than or equal to a given value.

Works with numeric types (Int, Decimal) and String fields (lexicographic comparison).

Syntax:

{ "fieldName": { "$gte": value } }

In a where clause:

{
  "from": "products",
  "where": {
    "price": { "$gte": 5.00 }
  }
}

This returns all products with a price of 5.00 or higher.

{
  "$gte": GetValueExpression or integer or integer or number or null or ArrayValue or $toString or boolean or string or Json
}

Availability: API v1 · API v2 (beta) — v2 form: $lt

Operator to check if a field is less than a given value.

Works with numeric types (Int, Decimal) and String fields (lexicographic comparison).

Syntax:

{ "fieldName": { "$lt": value } }

In a where clause:

{
  "from": "products",
  "where": {
    "price": { "$lt": 3.00 }
  }
}

This returns all products with a price less than 3.00.

{
  "$lt": GetValueExpression or integer or integer or number or null or ArrayValue or $toString or boolean or string or Json
}

Availability: API v1 · API v2 (beta) — v2 form: $lte

Operator to check if a field is less than or equal to a given value.

Works with numeric types (Int, Decimal) and String fields (lexicographic comparison).

Syntax:

{ "fieldName": { "$lte": value } }

In a where clause:

{
  "from": "products",
  "where": {
    "price": { "$lte": 10.00 }
  }
}

This returns all products with a price of 10.00 or less.

Range queries:

Combine with $gte for range queries:

{
  "from": "products",
  "where": {
    "$and": [
      { "price": { "$gte": 5.00 } },
      { "price": { "$lte": 15.00 } }
    ]
  }
}

{
  "$lte": GetValueExpression or integer or integer or number or null or ArrayValue or $toString or boolean or string or Json
}

Availability: API v1 · API v2 (beta) — v2 form: $has

Checks whether a field contains the specified feature (token).

$has is a low-level operation that works at the feature level. Features can differ significantly from the original data, especially for text fields when analyzers are used.

For example, if you have a field called content with the text "programmers and horses", the field would have features 'programmer' and 'hors' (stems produced by the English analyzer).

Syntax:

{ "fieldName": { "$has": "feature" } }

For Text fields:

{
  "from": "products",
  "where": {
    "description": { "$has": "coffee" }
  }
}

For array fields (String[]):

{
  "from": "products",
  "where": {
    "tags": { "$has": "organic" }
  }
}

This checks if the tags array contains the value "organic".

Difference from $match:

• $has matches exact features (stemmed tokens for Text fields)
• $match performs fuzzy text matching and handles multiple tokens

Use $has when you need precise feature matching, and $match for natural language search.

{
  "$has": GetValueExpression or integer or integer or number or null or ArrayValue or $toString or boolean or string or Json
}

Availability: API v1 · API v2 (beta) — v2 form: $defined

Operator to filter rows based on whether a nullable field has a value or is null.

Syntax:

{ "fieldName": { "$defined": true } }
{ "fieldName": { "$defined": false } }

Find rows where field has a value:

{
  "from": "products",
  "where": {
    "discount": { "$defined": true }
  }
}

Find rows where field is null:

{
  "from": "products",
  "where": {
    "discount": { "$defined": false }
  }
}

This is useful for filtering out incomplete data or finding records with missing values.

{
  "$defined": boolean
}

Availability: API v1 · API v2 (beta) — v2 form: $exists

An operator to check if specific features exist in a field.

Syntax:

{ "$exists": "fieldName" }
{ "$exists": ["field1", "field2"] }

Check if a field has any features:

{
  "from": "products",
  "where": {
    "$exists": "tags"
  }
}

This returns rows where the tags field has at least one feature (non-empty).

Check multiple fields:

{
  "from": "products",
  "where": {
    "$exists": ["name", "category"]
  }
}

This returns rows where both name and category have features.

{
  "$exists": PropositionSet
}

Availability: API v1 · API v2 (beta) — v2 form: $and

Performs a logical and operation on the given array containing two or more Propositions.

• With the non-inference query (e.g: Search, Similarity), the $and operator guarantees that all propositions are met. For instance, the following search query:

{
  "from": "products",
  "where": {
    "$and": [
      { "description": "super slim laptop" },
      { "price": { "$gt" : 200 } }
    ]
  }
}

will always find products of which description is super "slim laptop" and price is greater than 200

• With the inference query (e.g: Predict, Match, Recommend), the $and operator does not guarantee that all propositions are met. For instance, the following predict query:

{
  "from": "products",
  "where": {
    "$and": [
      { "description": "super slim laptop" },
      { "price": { "$gt" : 200 } }
    ]
  },
  "price": "tag"
}
Aito might look for products with a price greater than 200 but do not match the description of super slim laptop or products that match the description but do not meet the price condition. This is because there might be a lack of data (e.g: not enough products in the price range) to make a sophisticated prediction.

To guarantee that all propositions are met in a inference query, refer to $atomic

{
  "$and": [Proposition or PrimitiveProposition]
}

Availability: API v1 · API v2 (beta) — v2 form: $or

Performs a logical or operation on the given array containing two or more Propositions.

{
  "$or": [Proposition or PrimitiveProposition]
}

Availability: API v1 · API v2 (beta) — v2 form: $not

Performs a logical not operation on the given Proposition.

{
  "$not": Proposition or PrimitiveProposition
}

Availability: API v1 · API v2 (beta) — v2 form: $asc

Sort returned hits in ascending order (A-Z) based on the given attribute or custom scoring function.

{
  "$asc": Value
}

Availability: API v1 · API v2 (beta) — v2 form: $asc

Sort returned hits in ascending order (A-Z) based on the given attribute (or column).

{
  "$asc": string
}

Availability: API v1 · API v2 (beta) — v2 form: $desc

Sort returned hits in descending order (Z-A) based on the given attribute or custom scoring function.

{
  "$desc": Value
}

Availability: API v1 · API v2 (beta) — v2 form: $desc

Sort returned hits in descending (Z-A) order based on the given attribute (or column).

{
  "$desc": string
}

Availability: API v1 · API v2 (beta) — v2 form: $mod

Operator to check if the value of a field divided by a divisor has the specified remainder.

In other words perform a modulo operation. This operator supports object or array form. Note that the field will be converted to an integer (effectively a math floor) before the modulo operation.

{
  "$mod": ModPropositionObject or ModPropositionArray
}

Availability: API v1 · API v2 (beta) — v2 form: $multiply

Multiplication operation of given items.

{
  "$multiply": [Score]
}

Availability: API v1 · API v2 (beta) — v2 form: $divide

Division operation.

{
  "$divide": object or [Score]
}

Availability: API v1

Exponentiation operation. First item raised to the power of the second.

{
  "$pow": ExponentPropositionObject or ExponentPropositionArray
}

Availability: API v1 · API v2 (beta) — v2 form: $sum

Calculates sum of given items.

{
  "$sum": ContextValueQuery or [Score]
}

Availability: API v1 · API v2 (beta) — v2 form: $subtract

Subtraction operation.

{
  "$subtract": object or [Score]
}

Availability: API v1 · API v2 (beta)

Transforms a statement into a 'black box' proposition.

This prevents Aito from analyzing the proposition and using its parts separately in the statistical reasoning.

In practice the difference between normal 'white box' expressions, and the $atomic's black box expressions is: that the atomic expressions have a smaller bias, but a higher measurement error.

Consider the following example:

{
  "tags": "pen",
  "price": { "$gte": 200 } }
}

During the statistical reasoning: Aito may recognize that pens are often sold, and that over 200€ product purchases are somewhat common. As a result, Aito might assume the over 200€ pen to be a popular product.

Now, consider the expression:

{
  "$atomic": {
    "tags": "pen",
    "price": { "$gte" : 200 }
  }
}

The results of this expression will depend of the amount of data. If there are no over 200€ pens in the data: Aito will make no assumptions of the proposition's effect. On the other hand, if you have the data: Aito will recognize correctly, that the over 200€ pens are bought extremely rarely.

{
  "$atomic": Proposition
}

Availability: API v1 · API v2 (beta) — v2 form: $exact

Prevents featurization of the underlying proposition. Similar to $atomic, but even more precise: where $atomic keeps the proposition's parts together as one black-box statement, $exact also skips the field's analyzer, matching the raw stored value.

For example, on a Text field with an English analyzer:

{
  "query": { "$exact": "electric guitar" }
}

matches only rows where the query field contains exactly "electric guitar", treating it as a single proposition rather than decomposing it into the individual tokens ("electr", "guitar").

The value can be a plain value or a wrapped proposition:

{ "$exact": "exact phrase" }
{ "$exact": { "$is": "exact phrase" } }

{
  "$exact": Proposition or PrimitiveProposition
}

Availability: API v1 · API v2 (beta) — v2 form: $context

Provides ability to access the fields of the table specified in "from", instead of fields of the table in "get".

{
  "$context": Proposition
}

Availability: API v1

Provides ability to access the fields of the hit.

{
  "$hit": Score
}

Availability: API v1 · API v2 (beta) — v2 form: $on

$on operator is used to define conditional propositions or hard filters.

This is useful when you have limited amount of data and the condition would help to limit the context and provide better results. This can be done by providing a list containing of two items, the first object (or "prop") is the hypothesis and the second object (or "on") is the conditional.

In Aito the where clause contains propositions which aren't hard filters. Instead, Aito will turn all the propositions into features (the user's ID, every word in a text field, etc.). There are many of these and they are not statistically independent. Aito picks a subset of these features that are the best predictors of the field that is to be predicted. So what goes into the "where" is a description of the situation you're in and Aito tells you what you should expect to find if you look in a field. But the description is not taken at face value, Aito will ignore parts of it if it doesn't help the prediction.

However, there is another way to achieve this: the "$on" proposition. It is modeled after conditional probability. It is divided into two parts, the normal "where" parts and the conditional part ("hard filters"). The "$on" parameters explained:

{
  "from": "...",
  "where": {
    "$on": [
      {
        "message": "hello, world",
        "something": true,
        // other things you put in your "where" clause
      },
      {
        // The subset of data that exactly matches these conditions
        "userId": 42,
        "day": "monday"
      }
    ]
  },
  "predict": "..."
}

The $on can also be combined with normal query. If the $on condition is too strong, you could move parts of the filtering back to the where clause:

{
  "from": "...",
  "where": {
    "$on": [
      {
        "message": "hello, world",
        "something": true,
        // other things you put in your "where" clause
      },
      {
        // The subset of data that exactly matches these conditions
        "day": "monday"
      }
    ],
    "user_id": 42
  },
  "predict": "..."
}

{
  "$on": OnPropositionObject or OnPropositionArray
}

Availability: API v1 · API v2 (beta) — v2 form: $knn

The $knn operator is an adaptation of the classic k-nearest neighbor algorithm.

Aito's $knn operator identifies k most similar rows to the conditions defined in the 'near' parameter. The similarity metric is the same metric used in the similarity query. The k nearest rows can be used in inference.

The $knn operator can be useful in situation where there is no training data. For example:

{
  "from": "impressions",
  "where": {
    "product.name": "Columbian Coffee",
    "product.tags": "high quality coffee"
  },
  "predict": "purchase"
}

The query would not yield sensible results since there's no such product existed in the current data. This can be improved by using the $knn operator:

{
  "from": "impressions",
  "where": {
    "$knn": {
      "k": 5,
      "near": {
        "product.name": "Columbian Coffee",
        "product.tags": "high quality coffee"
      }
    }
  },
  "predict": "purchase"
}

In the query above, Aito would look for 5 entries that are most similar to the given criteria in "near" and use that for inference.

{
  "$knn": KnnPropositionObject or KnnPropositionArray
}

Availability: API v1 · API v2 (beta) — v2 form: $nn

The $nn operator is similar to the classic k-nearest neighbor algorithm, except that it matches a dynamic number of entries that are roughly the same as the specified proposition.

Aito's $nn operator identifies all rows that are roughly same to the conditions defined in the 'near' parameter. This group of parameters can be used in inference. This rough sameness is based on the same score used in the $sameness, and you can inspect the score of the matching values with the following query:

{
  "from": "rfps",
  "where": {
    "$nn": [{
      "question": "Does your company comply to ISO 27001?"
    }]
  },
  "orderBy": "$sameness"
}

$nn accepts also threshold parameter that can used to make matching stricter or looser like here:

{
  "from": "rfps",
  "where": {
    "$nn": [{
      "question": "Does your company comply to ISO 27001?"
    }, 0.5]
  },
  "orderBy": "$sameness"
}

The default threshold is 1.0.

An examples of using $nn in inference relates to question answering setting, where there is a desire to avoid false positive present in classic $knn or the default Bayesian inferences.

An example of using $nn for answering RFP question is following:

{
  "from": "rfps",
  "where": {
    "question": {
      "$nn": ["Does your company comply to ISO 27001?"]
    }
  },
  "predict": "answer"
}

This specific question will match similar question in the database. Still, because this question may also match questions like 'Does your comply with ISO 9001?', it makes sense to to also use the question's 'Does your comply with ISO 9001?' conditional features in inference like this:

{
  "from": "rfps",
  "where": {
    "question": {
      "$on": [
        "Does your company comply to ISO 27001?",
        {"$nn": ["Does your company comply to ISO 27001?"]}
      ],
      "$nn": ["Does your company comply to ISO 27001?"]
    }
  },
  "predict": "answer"
}

In this example, we $nn identifies a group of similar questions and uses these in the inference. At the same time $on structure allows the inference to see e.g. ISO 27001 as a separate feature inside this group. In this way, the system can focus on similar questions, while using individual features like ISO 27001 to infer the right answer.

{
  "$nn": NnPropositionObject or NnPropositionArray
}

Availability: API v1 · API v2 (beta) — v2 form: $numeric

Operator to check if a numeric field fuzzy matches a given number.

By default, numbers are compared exactly against one another. The $numeric proposition signifies that comparisons should be inexact and that the target is somewhere close to the specified number. The size of the region depends on the spread and density of the data.

{
  "$numeric": GetValueExpression or integer or integer or number or null
}

Availability: API v1 · API v2 (beta) — v2 form: $hash

$hash converts the field value into a hash integer.

The hash code can be used to split non-integer data pseudo-randomly in the evaluate query.

{
  "$hash": Proposition or PrimitiveProposition
}

Availability: API v1 · API v2 (beta)

$toString operator is used to convert a nummeric value to string.

This is useful when you want to use a numeric as input for an operator or a field that requires text input. For example:

{
  "description": {
    "$match": {
      "$toString": { "$get": "id" } 
    }
  }
}

{
  "$toString": GetValueExpression or integer or integer or number or null or ArrayValue or $toString or boolean or string or Json
}

Availability: API v1 · API v2 (beta) — v2 form: $p

"$p" can be used in the "orderBy" clause of the Generic query to get the most probable values. When used this way, it is similar to the Match query.

In the grocery demo dataset, running the following query would yield products with name similar to "lactose" that have the highest probabilities that it would be purchased:

{
  "from": "impressions",
  "where": {
    "product.name": {"$match": "lactose"}
  },
  "get": "purchase",
  "orderBy": "$p"
}

Similar to the Match query, running the following query would yield the most likely product based on all the fields of the linked product table:

{
  "from": "impressions",
  "where": {
    "context.user": "bob",
    "purchase": true
  },
  "get": "product",
  "orderBy": "$p"
}

Since the product field in the impressions table is linked to the products table, Aito would find all the statistical relations between what is declared inside the "where" clause and all the fields feature of a product, that is, id, name, category, price, tag. In this case, the probability score is the normalized product of the lift of each field's feature. We can investigate this by opening up the explanation adding the $why operator to the "select" clause (e.g: "select": ["$score", "$why"]):

"$why": {
  "type": "product",
  "factors": [
    {
      "type": "hitPropositionLift",
      "proposition": { "id" : 6410405093677 },
      "value": 1.9827806375460209,
      "factors": [
        {
          "type": "relatedPropositionLift",
          "proposition": { "purchase" : true },
          "value": 1.9827806375460209
        }
      ]
    },
    {
      "type": "hitPropositionLift",
      "proposition": { "$not" : { "name" : { "$has": "puikula" } } },
      "value": 1.0472308585357502,
      "factors": [
        {
          "type": "relatedPropositionLift",
          "proposition": { "purchase" : true },
          "value": 1.0472308585357502
        }
      ]
    }
    ...
  ]
}

We can see that the probability score is composed of lift of an id feature, a name feature and others.

See also $p and $lift.

Availability: API v1 · API v2 (beta) — v2 form: $f

"$f" can be used in the "orderBy" clause of the Generic query to get the frequency of a feature.

Availability: API v1 · API v2 (beta) — v2 form: $lift

"$lift" can be used in the "orderBy" clause of the Generic query to get the most likely values based on lifts of features with regard to other features.

In the grocery demo dataset, running the following query would yield products with name similar to "lactose" that have the highest lifts that it would be purchased:

{
  "from": "impressions",
  "where": {
    "product.name": {"$match": "lactose"}
  },
  "get": "purchase",
  "orderBy": "$lift"
}

Running the following query would yield the most likely product based on all the fields of the linked product table:

{
  "from": "impressions",
  "where": {
    "context.user": "bob",
    "purchase": true
  },
  "get": "product",
  "orderBy": "$lift"
}

Since the product field in the impressions table is linked to the products table, Aito would find all the statistical relations between what is declared inside the "where" clause and all the fields feature of a product, that is, id, name, category, price, tag. In this case, the lift score is the product of the lift of each field's feature. We can investigate this by opening up the explanation adding the $why operator to the "select" clause (e.g: "select": ["$score", "$why"]):

"$why": {
  "type": "product",
  "factors": [
    {
      "type": "hitPropositionLift",
      "proposition": { "id" : 6410405093677 },
      "value": 1.9827806375460209,
      "factors": [
        {
          "type": "relatedPropositionLift",
          "proposition": { "purchase" : true },
          "value": 1.9827806375460209
        }
      ]
    },
    {
      "type": "hitPropositionLift",
      "proposition": { "$not": { "name" : {"$has": "puikula" } } },
      "value": 1.0472308585357502,
      "factors": [
        {
          "type": "relatedPropositionLift",
          "proposition" : { "purchase" : true },
          "value": 1.0472308585357502
        }
      ]
    }
    ...
  ]
}

We can see that the lift score is composed of lift of an id feature, a name feature and others.

See also $p and $lift.

Availability: API v1 · API v2 (beta) — v2 form: $similarity

"$similarity" can be used in Generic query to get most similar rows based on the contents of the "where" clause.

Consider the following example. It will return all the products, that contain 'iphone' in the title. It also sorts the results by their similarity to the 'iphone' and highlight the 'iphone' term in the product title field.

{
  "from": "product",
  "where": { "title": { "$match": "iphone" } },
  "get": "message",
  "orderBy": "$similarity",
  "select": ["title", "$highlight"]
}

Availability: API v1 · API v2 (beta) — v2 form: $sameness

"$sameness" can be used in Generic query to get most roughly the same rows based on the contents of the "where" clause.

Consider the following example. It will return all the questions, that are roughly same as 'How can I order a sim card?' based on how closely they match the question.

{
  "from": "questions",
  "where": { "title": { "$nn": ["How can I order a sim card?"] } },
  "get": "message",
  "orderBy": "$sameness"
}

Availability: API v1

Provides ability to access the fields of the hit.

{
  "$hit": Score
}

Availability: API v1

Conceptually similar to the plain $p operator, but allows using a customized proposition for the probability score calculation.

This $p operator enables more options to customized the probability score calculation, especially when getting the values of linked table:

1. Narrow down the fields that are used to calculate the probability: This is similar to the behavior of the "basedOn" clause of the Match query When calculating the probability of a linked field, aito used all the fields of the linked table (See $p for how the probability is calculated for a linked field). If you would like to narrow down how the probability is calculated, you can add the field name following the $p. For example, find the most likely product based on only the product name:

{
  "from": "impressions",
  "where": {
    "context.user": "bob",
    "purchase": true
  },
  "get": "product",
  "orderBy": {
    "$p": "name"
  }
}

You can also calculate the probability based on multiple fields by using the array format. For instance:

{
  "$p": ["category", "tag"]
}

2. Calculate the probability based on a specific context: This is similar to the behavior of the Recommend query By combining with the $context operator, the probability score can be defined as the probability of a context. For instance, to find the products with the highest probability that the product would be purchased:

{
  "from": "impressions",
  "where": {
    "context.user": "bob"
  },
  "get": "product",
  "orderBy": {
    "$p": {"$context": {"purchase": true}}
  }
}

{
  "$p": PropositionSet
}

Availability: API v1

Declares the probability of the given proposition(s). This is a more configurable version of $p operation

Note the basedOn field can be used to reduce noise in recommendations. BasedOn contains the list of fields in hit table, that are used in the probability calculation. If basedOn is not defined, all fields are used, including the noisy ones.

Used in order by to sort the result by the probability in descending order.

{
  "$probability": object
}

Availability: API v1

Conceptually similar to the plain $lift operator, but allows using a customized proposition for the lift score calculation.

This $lift operator enables more options to customized the lift score calculation, especially when getting the values of linked table.

1. Narrow down the fields that are used to calculate the lift: This is similar to the behavior of the "basedOn" clause of the Match query When calculating the lift of a linked field, aito used all the fields of the linked table (See $lift for how the lift is calculated for a linked field). If you would like to narrow down how the lift is calculated, you can add the field name following the $lift. For example, find the most likely product based on only the product name:

{
  "from": "impressions",
  "where": {
    "context.user": "bob",
    "purchase": true
  },
  "get": "product",
  "orderBy": {
    "$lift": "name"
  }
}

You can also calculate the lift based on multiple fields by using the array format. For instance:

{
  "$lift": ["category", "tag"]
}

2. Calculate the lift based on a specific context: This is similar to the behavior of the Recommend query By combining with the $context operator, the lift score can be defined as the lift of a context. For instance, to find the products with the highest lift of getting purchased:

{
  "from": "impressions",
  "where": {
    "context.user": "bob"
  },
  "get": "product",
  "orderBy": {
    "$lift": {"$context": {"purchase": true}}
  }
}

{
  "$lift": PropositionSet
}

Availability: API v1

Declares the lift of the given proposition(s). This is more configurable version of $lift.

NOTE: This can be used if recommendations pick noise from irrelevant fields. Overall, using only relevant fields often makes inference both more accurate and faster.

Used in order by to sort the result by the probability in descending order.

{
  "$probabilityLift": object
}

Availability: API v1

The mean of given value in context

Example 1:

    {
      "$mean": {
        "$context": "score"
      }
    }
                            

{
  "$mean": ContextValueQuery
}

Availability: API v1

An empirical frequency based probability estimate

$freqP allows you to specify two fields, called f (success) and n (trials) and an additional field called p (prior probability).

$freqP will calculate probability estimate for each row based on the f (success), n (trials) and p (prior probability) fields. Priori probability is the probability of success before any data is observed and 0.5 by default.

$freqP comes also with variance information, which means that it can be used together with $decision operation to solve the multi-armed bandit problem.

{
  "$freqP": object
}

Availability: API v1

$decision-feature exist to solve the multi-armed bandit problem, where we have:

1. Decision options with uncertainty about the reward probabilities
2. And need to gather more data from the options to better estimate reward probabilities

This problem exist classically in advertisement, where there is uncertainity of the click-through rate of the ads. In such case, one needs to find a trade-off between exploring new ads and exploiting the best ads.

$decisions operation returns a 'decision' score, which can be used to randomly order options and select the first one(s). It implements a Bayesian solution, where the decision score is sampled from a normal distribution, that uses the mean and standard error of the given score.

NOTE: that $decision only works with scores having an inherent variance information. This variance is present right now only with numerical estimates calculated with $sum and $freqP. $decision is not yet supported for e.g. $p or basic probability calculation.

{
  "$decision": object
}

Availability: API v1 · API v2 (beta) — v2 form: $similarity

Conceptually similar to the plain $similarity operator, but allows using a customized proposition for the similarity score calculation.

The plain $similarity operator calculates the similarity score based on the "where" clause contents, whereas this $similarity operator calculates the similarity score based on the given proposition.

These Generic Queries would yield the same results:

{
  "from": "products",
  "where": {
    "name": {"$match": "coffee"}
  },
  "orderBy": "$similarity"
}

{
  "from": "products",
  "orderBy": {
    "$similarity": {
      "name": "coffee"
    }
  }
}

This $similarity operation is useful for customizing scoring as the example below. Please refer to GenericQuery query with custom scoring example.

{
  "from": "impressions",
  "where": {
    "context.user": "veronica"
  },
  "get": "product",
  "orderBy": {
    "$multiply": [
      {
        "$p": {
          "$context": {
            "purchase": true
          }
        }
      },
      {
        "$similarity": {
          "name": "coffee"
        }
      }
    ]
  }
}

{
  "$similarity": Proposition or PrimitiveProposition
}

Availability: API v1 · API v2 (beta) — v2 form: $sameness

This operator provides a score, which reflect whether the return entry is roughly the same as the given proposition / data.

E.g. if you have a phrase "How to order a sim card?" it should be judged roughly the same as "Ho can I order a sim card?". The values above 1 are considered to be roughly same and values under 1 are considered to be roughly distinct.

$sameness works in similar way to $similarity, except that it does more strict matching. E.g. query "sim card" provides above 1 similarity score with "How can I order sim card?", but it provides significantly under 1 sameness score. $sameness works better in situations, where a more restrictive is scoring is needed to avoid false matches. An example of this is e.g. question answering situation where, where one needs to match questions more strictly in order to avoid false positivess

{
  "$sameness": Proposition or PrimitiveProposition
}

Availability: API v1

This operator provides a score, which reflect whether the return entry is analogious with the given value in respect to some other values.

E.g. if you have a query word 'cheap' it should have a high $analogy score with words like affordable and inexpensive in respect to correct query result.

E.g. if you have a text "Netflix has great series" it should have a high $analogy score with in respect to a text like 'There are good movies' in respect to sentiment or feedback category.

{
  "$analogy": object
}

Availability: API v1

$normalize operator can be used in the "orderBy" clause of the Generic query to make a score to sum to 1. For example, you can normalize the $lift or the $lift object to 1:

{
  "from": "impressions",
  "where": {
    "product.name": {"$match": "lactose"}
  },
  "get": "purchase",
  "orderBy": {
    "$normalize": "$lift"
  }
}

or

{
	"from": "impressions",
	"where": {
		"context.user": "bob",
		"purchase": true
	},
	"get": "product",
	"orderBy": {
		"$normalize": {
      "$lift": { "$context": { "click": "true" } }
    }
	}
}

{
  "$normalize": Score
}

Availability: API v1

Impact of the scored valued (like product) to a value in the context (like star rating) relative to base. If base is not given, the value average is used instead.

{
  "$impact": object
}

Availability: API v1

The frequency / count of the given proposition in context.

{
  "$f": ContextPropositionQuery
}

Availability: API v1 · API v2 (beta) — v2 form: $mean

Mean or average. Can be used to calculate the average of a field. Similar to SQL AVERAGE operation.

{
  "$mean": Score
}

Availability: API v1 · API v2 (beta) — v2 form: $sum

Sum. Can be used to calculate the sum of the field.

{
  "$sum": Score
}

Availability: API v1

Count the frequency of cases the proposition / condition is true

{
  "$f": Proposition
}

Availability: API v1

Allows one to name the aggregate projection results

Availability: API v1

$index selects the insertion index of a row. It can be used together with $mod to select parts of a table. It's useful for example in Evaluate query for selecting training or test data.

Availability: API v1 · API v2 (beta) — v2 form: $why

Configurable explanations. Allows providing explanation specific highlights and matching.

Note: that why will contain highlights / matches also for the where clause.

{
  "$why": object
}

Availability: API v1 · API v2 (beta) — v2 form: $highlight

Sometimes, there is a need to highlight fields or individual fields' words in the examined hit. This can happen e.g. if the user is searching for specific term. Highlight can also be used in inference situations. E.g. when providing personalized search results, if the user has lactose-intolerance, it can be useful to highlight both 'lactose-free' as a positive factor and 'lactose' as a negative one.

The $highlight will provide for hit each field with HTML escaping and highlighting of positive and negative details.

  {
    "from" : "impressions",
    "where" : {
      "context.query" : "bread"
    },
    "recommend" : "product",
    "goal"      : {"purchase": true},
    "select"    : ["$p", "name", "$highlight"],
    "limit"     : 1
  }

This query produces following result containing a $highlight field:

{
  "offset": 0,
  "total": 42,
  "hits": [
    {
      "$p": 0.5967894477818738,
      "name": "VAASAN Ruispalat 660g 12 pcs fullcorn rye bread",
      "$highlight": [
        {
          "score": 2.5780035230413474,
          "field": "name",
          "highlight": "VAASAN Ruispalat 660g 12 pcs fullcorn rye <font color=\"green\">bread</font>"
        },
        {
          "score": 2.3703213948674122,
          "field": "tags",
          "highlight": "<font color=\"green\">gluten</font> bread"
        },
        {
          "score": 4.9484160228241825,
          "field": "$context.context.query",
          "highlight": "<font color=\"green\">bread</font>"
        }
      ]
    }
  ]
}

The $highlight fields contains a list of higlights. Eech highlight specifies highlights relative importance in the score field, the field that was relevant for the hit and the field content highlighted and html encoded. Note that also the 'where' clause content is highlighted under the $context. prefix.

Availability: API v1 · API v2 (beta) — v2 form: $highlight

Parametric highlight makes it possible to highlight the most important parts of the query that contributed to the score in more configurable way. This is useful for debugging and understanding the scoring process.

The highlight is returned as a list of strings, where each string represents a part of the query that contributed to the score. The strings are ordered by importance, with the most important part first.

The highlight is only returned if the query contains at least one term that contributed to the score. If the query contains no terms that contributed to the score, the highlight is not returned.

NOTE: That highlight also highlights the fields in 'where' clause.

Parametric highlight allows user to configure positive highlight and negative highlight html tags. One can also configure the highlight sensitivity, e.g. that how positive or negative the score has to be to trigger highlight. Highlighted fields are HTML encoded by default, but this can be also disabled via parameter.

{
  "$highlight": object
}

Availability: API v1 · API v2 (beta) — v2 form: $matches

Sometimes, there is a need to highlight fields or individual fields' words in the examined hit. This can happen e.g. if the user is searching for specific term. Highlight can also be used in inference situations. E.g. when providing personalized search results, if the user has lactose-intolerance, it can be useful to highlight both 'lactose-free' as a positive factor and 'lactose' as a negative one.

The $matches will provide for hit each field information about how the field is related to the hit score. It's worth emphasizing that the $matches is based on the hit score. This means that the matches will contain only the features that have contributed to the hit score. So the score is the probability, the $matches will contain items, that contributed to the probability. If the score is based on similarity, the $matches will contain the features specified inside the similarity query. This means that e.g. $match operations in where clause will not be included in the $matches.

Consider the following recommend query:

  {
    "from" : "impressions",
    "where" : {
      "context.user": "larry"
    },
    "recommend" : "product",
    "goal"      : {"purchase": true},
    "select"    : ["$score", "name", "$matches"],
    "limit"     : 1
  }

This query produces following result containing a $matches field:

{
    "offset" : 0,
    "total" : 42,
    "hits" : [ {
      "$score" : 0.1346800331844607,
      "name" : "Vaasan Ruispalat thin sliced rye bread 6pcs/195g",
      "$matches" : {
        "name" : [ {
          "begin" : 33,
          "end" : 38,
          "feature" : "bread",
          "text" : "bread",
          "why" : {
            "type" : "product",
            "factors" : [ {
              "type" : "hitPropositionLift",
              "proposition" : {
                "name" : {
                  "$has" : "bread"
                }
              },
              "value" : 2.658334404390319,
              "factors" : [ {
                "type" : "relatedPropositionLift",
                "proposition" : {
                  "context.user" : {
                    "$has" : "larry"
                  }
                },
                "value" : 2.658334404390319
              } ]
            } ]
          },
          "score" : 2.658334404390319
        } ],
        "tags" : [ {
          "begin" : 0,
          "end" : 6,
          "feature" : "gluten",
          "text" : "gluten",
          "why" : {
            "type" : "product",
            "factors" : [ {
              "type" : "hitPropositionLift",
              "proposition" : {
                "tags" : {
                  "$has" : "gluten"
                }
              },
              "value" : 2.184491633564294,
              "factors" : [ {
                "type" : "relatedPropositionLift",
                "proposition" : {
                  "context.user" : {
                    "$has" : "larry"
                  }
                },
                "value" : 2.184491633564294
              } ]
            } ]
          },
          "score" : 2.184491633564294
        } ]
      }
    } ]
  }

The $matches contains separate match list for 'name' and 'tags' fields. Both fields have one match. 'Name' contains a match 'bread' as larry seems to be purchasing bread 2.67 times more often than average user. Both matches contain the begin and end positions, that identify the location of the match in the field's text. These, alongside the text field,that contains the matching original text, can be used to highlight or bold the matching statement in UI. The $why field contains the explanation for the match, and it traces the score batch to the user variable.

Feature field contains the matched feature, and it can be used inside $has proposition to find other content with the same feature.

Availability: API v1 · API v2 (beta) — v2 form: $matches

A version of $matches, that accepts parameters and can could be configured, except that it doesn't yet have any configuration parameters.

NOTE: That matches will also match the fields in the 'where' clause.

{
  "$matches": object
}

Availability: API v1

Named projection makes it possible to name the projected values.

This is useful for e.g. naming different averages with more descriptive names as in the following example:

{
  "ctr": {"$mean": { "$context": "click" } },
  "meanScore": {"$mean": { "$context": "score" } }
}

Availability: API v1

$index is a built-in variable which indicates the insertion index of a row. It can be used together with $mod to select parts of a table. It's useful for example in Evaluate query for selecting training or test data.

Availability: API v1

$sort is a built-in field that can be used to access the sort value used in the orderBy-clause.

Availability: API v1 · API v2 (beta) — v2 form: $score

$score is a built-in field that can be used to access the sort value used in the orderBy-clause, when the sort-value is a numeric score like a probability.

Availability: API v1 · API v2 (beta) — v2 form: $p

$p is a built-in field that can be used to access the value used by orderBy-clause, when the sort-value is a probability. See $p for more information.

Availability: API v1 · API v2 (beta) — v2 form: $why

When selecting $why, Aito opens up why a certain result was predicted. Explanation contains 4 different factors, which are explained below.

The factors are for an estimate of form:

<InlineMath math="p(x_i | A, B, C)" />

"baseP"

The base probability.

<InlineMath math="p(X)" />

"normalizer"

Aito has two different normalizes, that are

• exclusiveness normalizer
• trueFalseExclusiveness normalizer

These normalizes are often grouped into a single 'product' component.

{
  "type" : "product",
  "factors" : [ {
    "type" : "normalizer",
    "name" : "exclusiveness",
    "value" : 1.0119918068684681
  }, {
    "type" : "normalizer",
    "name" : "trueFalseExclusiveness",
    "value" : 1.09917613448721
  } ]
}

The exclusiveness normalizer is only used, when exclusiveness is on. In this case, it is assumed that only one feature can be true at the same time, and that one feature will be true. In practice, exclusiveness enforces the probabilities of alternative features to sum to 1.0.

The normalizer is of form:

<InlineMath math="\\dfrac{1}{sum((p(X_0) + p(X_1) + ...))}" />

Aito makes a probability estimation for both X and ¬X on the background and uses the trueFalseExclusiveness normalizer to assert that the probabilities P(X) and P(¬X) sum to 1.0.

The normalizer is of form:

<InlineMath math="\\dfrac{1}{p(X) + p(\\neg X)}" />

"relatedVariableLift"

Probability lifts. For example: the lift may say a product is clicked with 2.3x likelihood (or 130% higher likelihood), when it has 5 stars.

A probability lift is of form:

<InlineMath math="\\dfrac{p(A | X)}{p(A)}" />

"calibration"

Aito calibrates the final probability so that its confidence reflects the amount of statistical evidence behind the prediction. Whenever the displayed $p differs from the raw product of the factors above, the difference is shown as an explicit calibration factor, so the $why breakdown always multiplies out to the displayed $p.

{
  "type" : "calibration",
  "name" : "support-tempering(auto)",
  "value" : 0.61
}

Two calibrations can appear (also combined into one factor when both are active):

• support-tempering(auto) — on by default. Each prediction's evidence is tempered by its effective statistical support: predictions backed by many distinct training examples pass through essentially unchanged (value ≈ 1.0), while predictions whose evidence rests on few or heavily overlapping examples are softened towards the prior. Deterministic lookups and $knn/$nn style matches are recognised as regular, well-supported evidence and are not softened. Opt out with aito.neffTemper.enabled=false (see the deployment configuration reference).
• temperature(τ=…) — a fitted global temperature from query-time calibration (config.calibrate), applied on top of support-tempering and fitting only the residual mis-calibration.

Availability: API v1 · API v2 (beta) — v2 form: $value

$value is a built-in field which contains the value of the returned object. $value can be used to access the field value referred in the predict, match, recommend and get-clauses, when the returned item is either a field value or a field feature/proposition. $value is intended to replace the 'feature' field in the long term.

The $value field has been added to contain the information in the ‘feature’ so that for query:

  {
    "from" : "products",
    "where" : {
      "title" : "apple iphone"
    },
    "predict": "tags",
    "select" : ["$p", "$value"],
    "limit":3
  }

The result will be the following:

  {
    "offset" : 0,
    "total" : 10,
    "hits" : [ {
      "$p" : 0.3656914544001758,
      "$value" : "premium"
    }, {
      "$p" : 0.1546922568903658,
      "$value" : "cover"
    }, {
      "$p" : 0.09493670104339776,
      "$value" : "macosx"
    } ]
  }

Value works similarly, when predicting the field value, using the generic query.

  {
    "from" : "products",
    "where" : {
      "title" : "apple iphone"
    },
    "get": "tags",
    "orderBy" : "$p",
    "select" : ["$p", "$value"],
    "limit":3
  }

Or when when predicting the field features with the generic query:

  {
    "from" : "products",
    "where" : {
      "title" : "apple iphone"
    },
    "get": "tags.$feature",
    "orderBy" : "$p",
    "select" : ["$p", "$value"],
    "limit":3
  }

Availability: API v1

$proposition is a built-in field which contains the proposition object of the returned feature. The returned proposition is compatible with the proposition format and it can be used as such in the where clause.

Consider the following query:

  {
    "from": "products",
    "where": {
      "title": "Apple"
    },
    "predict": {
      "$on": [
        { "$exists": "tags" },
        { "$and": [
          { "tags": { "$match": "phone" } },
          { "$not": { "tags": { "$match": "laptop" } } }
        ] }
      ]
    },
    "select": ["$p", "$value", "$proposition"],
    "limit": 1
  }

This provides the following results:

  {
    "offset" : 0,
    "total" : 10,
    "hits" : [ {
      "$p" : 0.22622976807854914,
      "$value" : "phone",
      "$proposition" : {
        "$on" : [ {
          "tags" : {
            "$has" : "phone"
          }
        }, {
          "$and" : [ {
            "tags" : {
              "$has" : "phone"
            }
          }, {
            "$not" : {
              "tags" : {
                "$has" : "laptop"
              }
            }
          } ]
        } ]
      }
    } ]
  }

Availability: API v1

Explain how a summation was calculated.

The SumExplanation most commonly appears when opening up the explanation (e.g: using the $why operator) of a score calculated using the $sum operator.

{
  "type": string,
  "terms": [ScoreExplanation]
}

Availability: API v1

This explanation object explains how a substraction result was calculated. It occurs in $why results, if $subtract operation is used.

{
  "type": string,
  "minuend": ScoreExplanation,
  "subtrahend": ScoreExplanation
}

Availability: API v1

This explanation object explains how a product score was calculated. It occurs in $why results, if $multiply operation is used.

The ProductExplanation most commonly appears when opening up the explanation (e.g: using the $why operator) of:

1. Aggregated score by product. For example, in a Match query:

{
  "from": "impressions",
  "where": {
    "context.user": "larry"
  },
  "match": "product",
  "select": ["$score", "name", "$why"]
}

The final score is a product of multiple score components:

{
  "type": "product",
  "factors": [
    {
      "type": "hitPropositionLift",
      "proposition": { "id" : 6410405216120 },
      "value": 599.5491890842981,
      "factors": [
        {
          "type": "baseLift",
          "value": 265.0
        },
        {
          "type": "relatedPropositionLift",
          "proposition": { "context.user" : "larry" },
          "value": 2.2624497701294266
        }
      ]
    },
    ...
  ]
}

2. A score calculated using the $multiply operator.

{
  "type": string,
  "factors": [ScoreExplanation]
}

Availability: API v1

Conceptually similar to BaseProbabilityExplanation but show the prior lift instead of prior probability.

See more Probability vs. Lift

{
  "type": string,
  "value": number
}

Availability: API v1

This explanation object explains how a division result was calculated. It occurs in $why results, if $divide operation is used.

{
  "type": string,
  "dividend": ScoreExplanation,
  "divisor": ScoreExplanation
}

Availability: API v1

Explain the initial weight of a feature. It can be understand as the prior probability <InlineMath math="p(X)" /> of a feature.

Let's take a look at an example of a Predict query:

{
  "from": "products",
  "where": {
    "name": "Columbian coffee"
  },
  "predict": "tags",
  "select": ["$p", "feature", "$why"]
}

When opening up the explanation with "$why" operator, a tag's feature "coffee" has a BaseProbabilityExplanation:

{
  "type": "baseP",
  "value": 0.16
}

This explanation tells that Aito gives the feature "coffee" a prior probability of 0.16.

{
  "type": string,
  "value": number,
  "proposition": PropositionExplanation
}

Availability: API v1

Explain how a related variable's lift was calculated.

A related variable (feature) most commonly appears when doing inference with some conditions. The RelatedVariableLiftExplanation explains how a variable of the conditions affecting the lift of a hit's variable.

Let's take a look at an example of Match query:

{
  "from": "impressions",
  "where": {
    "context.user": "larry"
  },
  "match": "product",
  "select": ["$score", "name", "$why"]
}

When opening up the explanation with "$why" operator, the first hit has an explanation as follows:

{
  "type": "hitVariableLift",
  "variable": "id:6410405216120",
  "value": 599.5491890842981,
  "factors": [
    {
      "type": "baseLift",
      "value": 265.0
    },
    {
      "type": "relatedVariableLift",
      "variable": "context.user:larry",
      "value": 2.2624497701294266
    }
  ]
}

This explains that the feature "context.user:larry" extracted from the conditions "where": { "context.user": "larry" } enhances the likelihood that the product having an id of 6410405216120 with a lift of 2.2624497701294266.

{
  "type": string,
  "proposition": PropositionExplanation,
  "value": number
}

Availability: API v1

Explain how a propositions's lift was calculated.

HitLinkPropositionLiftExplanation explains the impact of the value that links to table containing the returned hits.

Let's consider an example, where there is an impression table that has a numeric field 'product' that links to the product table. In such a case the HitLinkPropositionLift would explain the significance of the field 'product' in the impression table. E.g., if the product link's value is 4, the HitLinkPropositionLiftExplanation will explain the effect of the proposition { "product" : 4 }. If the value is 2.0, it means that the 4 product is estimated to be twice as probable just based on the statistics of the linking column.

{
  "type": string,
  "proposition": PropositionExplanation,
  "value": number
}

Availability: API v1

This format is essentially a normal score explanation, but it can contain additional $matches and $highlight fields. It is returned when ParametricWhy is used.

Availability: API v1

Explain how a propositions's lift was calculated.

A hit score was calculated by aggregating the score of its propositions (features). The HitPropositionLiftExplanation explains how different proposition was calculated.

A HitPropositionLift can be:

1. A similarity score A hit's field can contain a word that match the stem of the given similarity condition. That word would have a HitPropositionLift that is a similarity score. Let's take a look at an example of Similarity query:

{
  "from": "products",
  "similarity": {
    "name": "Columbian coffee",
    "tags": "expansive coffee"
  },
  "select": ["$score", "name", "tags", "$why"]
}

When opening up the explanation with "$why" operator, we can see that a hit with name "Juhla Mokka coffee 500g sj" containing the word coffee has a HitPropositionLiftExplanation as follows:

{
  "type": "hitPropositionLift",
  "proposition": "name:coffe",
  "value": 2.1726635013471625,
  "factors": [
    {
      "type": "exponent",
      "value": 2.1726635013471625,
      "base": {
        "type": "idf",
        "value": 2.1726635013471625
      },
      "power": {
        "type": "tf",
        "value": 1.0
      }
    }
  ]
}

2. An aggregated score of BaseLift and RelatedPropositionLift

Let's take a look at an example of Match query:

{
  "from": "impressions",
  "where": {
    "context.user": "larry"
  },
  "match": "product",
  "select": ["$score", "name", "$why"]
}

When opening up the explanation with "$why" operator, the first hit has a HitPropositionLiftExplanation as follows:

{
  "type": "hitPropositionLift",
  "proposition": { "id" : { "$has" : 6410405216120 } },
  "value": 599.5491890842981,
  "factors": [
    {
      "type": "baseLift",
      "value": 265.0
    },
    {
      "type": "relatedPropositionLift",
      "proposition": { "context.user": { "$has" : "larry" } },
      "value": 2.2624497701294266
    }
  ]
}

This explains that the initial lift of the feature "id:6410405216120" is 265 and when the user is Larray, the relatedPropositionLift is 2.2624497701294266. Hence the aggregated lift is <InlineMath math="265 * 2.2624497701294266 = 599.5491890842981" />

{
  "type": string,
  "proposition": PropositionExplanation,
  "value": number,
  "factors": [ScoreExplanation]
}

Availability: API v1

Default value explanation describes a constant value, typically given by the user.

{
  "type": string,
  "value": number
}

Availability: API v1

Default value explanation describes the default score for some operation.

For example TF-IDF scoring assigns default lift of 1.0 for all rows without matching terms.

{
  "type": string,
  "value": number
}

Availability: API v1

Explain how a special named score was calculated.

The NamedExplanation appears in two places:

• when calculating a score with exclusiveness, it explains the normalizer that enforces the probabilities of a feature to have sum of 1.0.
• when the displayed probability has been calibrated, it explains the calibration factor — support-tempering(auto) (on by default; confidence tempered by the effective statistical support of the evidence) and/or temperature(τ=…) (a fitted query-time calibration). See the $why documentation for details.

{
  "type": string,
  "name": string,
  "value": number
}

Availability: API v1

Explain how a probability was calculated.

The PredictExplanation most commonly appears when opening up the explanation (e.g: using the $why operator) of a Predict query. Let's take a look at an example of Predict query:

  {
    "from": "products",
    "where": {
      "name": "Columbian coffee"
    },
    "predict": "tags",
    "select": ["$p", "feature", "$why"],
    "limit": 22
  }

The first hit has an explanation of"

{
  "type": "product",
  "factors": [
    {
      "type": "baseP",
      "value": 0.16
    },
    {
      "type" : "product",
      "factors" : [ 
        {
          "type" : "normalizer",
          "name" : "exclusiveness",
          "value" : 1.0119918068684681
        }, 
        {
          "type" : "normalizer",
          "name" : "trueFalseExclusiveness",
          "value" : 1.09917613448721
        } 
      ]
    },
    {
      "type": "relatedVariableLift",
      "variable": "name:coffe",
      "value": 8.45603245079726
    }
  ]
}

{
  "type": string,
  "factors": [ScoreExplanation]
}

Availability: API v1

Explain how an exponent score was calculated.

The ExponentExplanation most commonly appears when opening up the explanation (e.g: using the $why operator) of an exponent score such as:

1. The tf-idf score to calculate the similarity in the Similarity query.
2. The score of the $pow operator.

{
  "type": string,
  "value": number,
  "base": ScoreExplanation,
  "power": ScoreExplanation
}

Availability: API v1

Explain the term frequency score.

The term frequency score is one component of the term frequency-inverse document frequency score which is used in Aito's similarity metrics.

{
  "type": string,
  "value": number
}

Availability: API v1

Explain the inverse document frequency score.

The inverse document frequency score is one component of the term frequency-inverse document frequency score which is used in Aito's similarity metrics.

{
  "type": string,
  "value": number
}

Availability: API v1

Explain how a field score was calculated.

The field explanation most commonly appears when opening up the explanation (e.g: using the $why operator) of a score that was calculated using:

1. A field value

{
  "from" : "impressions",
  "where" : {
    "product.name":{"$match": "coffee"}
  },
  "get":"product",
  "orderBy" : {
    "$multiply": ["$p", "price"]
  },
  "select": ["$score", "$why"]
}

The explanations would contains the value of the "price" field that was use in the $multiply operator.

{
  "type": "field",
  "field": "price",
  "value": 3.95
}

2. A field feature (e.g: $f operator for frequency):

{
  "from" : "impressions",
  "where" : {
    "product.name":{"$match": "coffee"}
  },
  "get":"product",
  "orderBy" : "$f",
  "select": ["$score", "$why"]
}

The explanation would contains the frequency of the feature.

{
  "type": "field",
  "field": "$f",
  "value": 152.0
}

{
  "type": string,
  "field": string,
  "value": number
}

Availability: API v1

Explain how a score was calculated.

Availability: API v1

FieldPropositionExpanation expresses a statement about a document field

For example the expression

{
  "tags": { "$has" : "laptop" }
}

states, that the tags field contains the "laptop" feature

{
  "fieldName": PropositionExplanation
}

Availability: API v1

This format is used in the $why and relate explanations for the '$has'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$has": IntArray or LongArray or DecimalArray or StringArray or BooleanArray or object or object or object or object or object or integer or integer or number or boolean or null or string or object
}

Availability: API v1

This format is used in the $why and relate explanations for the '$and'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$and": [PropositionExplanation]
}

Availability: API v1

This format is used in the $why and relate explanations for the '$or'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$or": [PropositionExplanation]
}

Availability: API v1

This format is used in the $why and relate explanations for the '$on'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$on": [PropositionExplanation]
}

Availability: API v1

This format is used in the $why and relate explanations for the '$not'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$not": PropositionExplanation
}

Availability: API v1

This format is used in the $why and relate explanations for the '$startsWith'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$startsWith": string
}

Availability: API v1

This format is used in the $why and relate explanations for the '$gt'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$gt": IntArray or LongArray or DecimalArray or StringArray or BooleanArray or object or object or object or object or object or integer or integer or number or boolean or null or string or object
}

Availability: API v1

This format is used in the $why and relate explanations for the '$gte'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$gte": IntArray or LongArray or DecimalArray or StringArray or BooleanArray or object or object or object or object or object or integer or integer or number or boolean or null or string or object
}

Availability: API v1

This format is used in the $why and relate explanations for the '$lt'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$lt": IntArray or LongArray or DecimalArray or StringArray or BooleanArray or object or object or object or object or object or integer or integer or number or boolean or null or string or object
}

Availability: API v1

This format is used in the $why and relate explanations for the '$lte'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$lte": IntArray or LongArray or DecimalArray or StringArray or BooleanArray or object or object or object or object or object or integer or integer or number or boolean or null or string or object
}

Availability: API v1

This format is used in the $why and relate explanations for the '$defined'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$defined": boolean
}

Availability: API v1

This format is used in the $why and relate explanations for the '$numeric'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$numeric": number
}

Availability: API v1

This format is used in the $why and relate explanations for the '$knn'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$knn": object
}

Availability: API v1

This format is used in the $why and relate explanations for the '$nn'-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

{
  "$nn": object
}

Availability: API v1

This format is used in the $why and relate explanations for the is-proposition. Explanations of this format follow the normal proposition syntax and it can be reused in the where-clause.

Availability: API v1

This is the format for all propositions used in the $why and relate explanations.

Availability: API v1

Name of a column in a table. Links are supported.

Availability: API v1

A Generic query to be evaluated in the Evaluate query

{
  "from": From,
  "where": Proposition,
  "get": Get,
  "orderBy": OrderBy,
  "select": Projection,
  "offset": integer,
  "limit": integer,
  "config": QueryConfig
}

Availability: API v1

Supported query to be evaluated in EvaluateGroupedQuery. Currently only support Generic query and Recommend query

Availability: API v1

The EvaluateGroupedQuery is similar to the EvaluateQuery with an addition option to group multiple entries into a single test case.

For example, if there exists a "customerCohort" identifier in "impressions" table, we can evaluate by the customerCohort instead of the individual customer with the following EvaluateGroupedQuery:

{
  "evaluate": {
    "from": "impressions",
    "where": {
      "customer": { "$get": "customer" }
    },
    "recommend": "product",
    "goal": { "purchase": true }
  },
  "group": "customerCohort",
  "test": {
    "customerCohort": { "$gte": 5 }
  },
  "select": ["trainSamples", "testSamples", "meanRank"]
}

{
  "train": Proposition,
  "test": Proposition,
  "testSource": TestSource,
  "select": Selection,
  "maxTime": integer,
  "group": string,
  "goal": Proposition,
  "evaluate": EvaluateGroupedOperation
}

Availability: API v1

A Match query to be evaluated in the Match query

{
  "from": From,
  "where": Proposition,
  "select": Projection,
  "match": Get,
  "basedOn": PropositionSet,
  "offset": integer,
  "limit": integer,
  "config": QueryConfig
}

Availability: API v1

The Generic Query to be evaluated in a EvaluateGroupedQuery

{
  "from": From,
  "where": Proposition,
  "get": Get,
  "orderBy": OrderBy,
  "select": Projection,
  "offset": integer,
  "limit": integer
}

Availability: API v1

Operation to be evaluated.

Availability: API v1

A Predict query to be evaluated in the Evaluate query

{
  "from": From,
  "where": Proposition,
  "predict": PropositionSet,
  "basedOn": PropositionSet,
  "exclusiveness": boolean,
  "select": Projection,
  "offset": integer,
  "limit": integer,
  "config": QueryConfig
}

Availability: API v1

A query to evaluate:

• Match query
• Predict query
• Similarity query
• Generic query

{
  "train": Proposition,
  "test": Proposition,
  "testSource": TestSource,
  "select": Selection,
  "maxTime": integer,
  "evaluate": EvaluateOperation
}

Availability: API v1

A Recommend query to be evaluated in the Evaluate query

{
  "from": From,
  "where": Proposition,
  "recommend": Get,
  "goal": Proposition,
  "select": Projection,
  "offset": integer,
  "limit": integer
}

Availability: API v1

A Similarity query to be evaluated in the Evaluate query

{
  "from": From,
  "where": Proposition,
  "get": Get,
  "similarity": Proposition or PrimitiveProposition,
  "select": Projection,
  "offset": integer,
  "limit": integer
}

Availability: API v1

Define the base and the exponent of the $pow operator in the array format.

The first item of the array is the base and the second item of the array is the exponent.