Concept learning update migration guide

Documentation

March 13, 2020

Aito is rolling the concept learning update starting March 23rd. This update includes major improvements in Aito's statistical reasoning, bug fixes, performance improvements and API changes. You can find more information about the update here.

The update also introduces two API incompatibilities. This guide describes the changes and the migration.

Changes in Explanation Objects ($why)

Consider the following query:

{
  "from" : "invoices",
  "where" : {
    "Item_Description" : "Store Management Rent"
  },
  "predict" : "Product_Category",
  "select" : ["$p", "feature", "$why"],
  "limit" : 1
}

Previously, Aito would return the following result:

{
  "offset": 0,
  "total": 36,
  "hits": [
    {
      "$p": 0.9995717068809318,
      "feature": "CLASS-1274",
      "$why": {
        "type": "product",
        "factors": [
          {
            "type": "baseP",
            "value": 0.17761033369214208
          },
          {
            "type": "normalizer",
            "name": "exclusiveness",
            "value": 0.999573491370787
          },
          {
            "type": "relatedVariableLift",
            "variable": "Item_Description:rent",
            "value": 5.641011031293095
          },
          {
            "type": "relatedVariableLift",
            "variable": "Item_Description:manag",
            "value": 5.410543091141314
          },
          {
            "type": "relatedVariableLift",
            "variable": "Item_Description:store",
            "value": 3.0082144959094568
          }
        ]
      }
    }
  ]
}

Now Aito returns the following result:

{
  "offset": 0,
  "total": 36,
  "hits": [
    {
      "$p": 0.9753500064252928,
      "feature": "CLASS-1274",
      "$why": {
        "type": "product",
        "factors": [
          {
            "type": "baseP",
            "value": 0.1761870760442699,
            "proposition": {
              "Product_Category": {
                "$has": "CLASS-1274"
              }
            }
          },
          {
            "type": "normalizer",
            "name": "exclusiveness",
            "value": 0.9763895461514779
          },
          {
            "type": "relatedPropositionLift",
            "proposition": {
              "$and": [
                {
                  "Item_Description": {
                    "$has": "rent"
                  }
                },
                {
                  "Item_Description": {
                    "$has": "store"
                  }
                },
                {
                  "Item_Description": {
                    "$has": "manag"
                  }
                }
              ]
            },
            "value": 5.6353753851278805
          }
        ]
      }
    }
  ]
}

The new format has following main changes:

  1. BaseP-component now contains the 'proposition'-field, which contains the predicted value's proposition. In recommend queries the BaseP-component's proposition field will contain the goal proposition.

  2. The explanation component type names have been renamed according to the following list

    1. "relatedVariableLift" ->  "relatedPropositionLift"

    2. "hitVariableLift" ->  "hitPropositionLift". "hitPropositionLift" occurs in $why of Match Query

    3. New "hitLinkPropositionLift" proposition is added to explain the effect of the predicted link variable in Match Query

  3. The explanation components no more have 'variable' field containing a string, but a 'proposition' field containing a proposition object.

So in essence, the '$why' component no more contains components of form:

{
  "type": "relatedVariableLift",
  "variable": "Item_Description:rent",
  "value": 5.641011031293095
}

Instead it contains components of form:

{
  "type": "relatedPropositionLift",
  "proposition": {
    "Item_Description": {
      "$has": "rent"
    }
  },
  "value": 5.635622992714686
}

So in order to do the migration, you need to look for 'relatedPropositionLift' and 'hitPropositionLift' component type identifiers instead of 'hitVariableLift' and 'hitPropositionLift' identifiers. Also, you need to parse proposition objects, instread of proposition strings. You can find more information about the proposition objects at the end of this guide.

Changes in Relate Query Result

Consider the following relate query:

{
  "from" : "invoices",
  "where" : {
    "Item_Description" : "Rent"
  },
  "relate" : { "Product_Category" : "CLASS-1274" },
  "select" : ["related", "lift", "condition"]
}

Previously, Aito provided the following response:

{
  "offset": 0,
  "total": 1,
  "hits": [
    {
      "related": "Product_Category:CLASS-1274",
      "lift": 5.641011031293095,
      "condition": "Item_Description:rent"
    }
  ]
}

Now Aito provides this response:

{
  "offset": 0,
  "total": 1,
  "hits": [
    {
      "related": {
        "Product_Category": {
          "$has": "CLASS-1274"
        }
      },
      "lift": 5.635622992714686,
      "condition": {
        "Item_Description": {
          "$has": "rent"
        }
      }
    }
  ]
}

Previously 'related' and 'condition' fields would contain strings identifying the propositions. Now the 'related' and 'condition' fields contain proposition objects identifying the proposition.

To do the migration, you need to parse the 'related' and 'condition' fields as proposition objects, instead of parsing them as proposition strings. There is more information about the proposition objects at the end of this chapter.

The proposition object format:

So previously the propositions were identified with strings like:

"Item_Description:rent"

Such strings been replaced with proposition objects like:

{
  "Item_Description": {
    "$has": "rent"
  }
}

A proposition object can be complex:

{
  "$and": [
    {
      "Item_Description": {
        "$has": "rent"
      }
    },
    {
      "Item_Description": {
        "$has": "store"
      }
    },
    {
      "Item_Description": {
        "$has": "manag"
      }
    }
  ]
}

Both "$why" explanation objects and "relate" results may contain complex propositions in cases where complex propositions were presented in the "where" clause. "$why" field may also contain complex propositions in cases where the concept learning has formed complex expressions to do higher-level statistical reasoning.

It is guaranteed that the proposition objects can be reused in the where clause, and in place of propositions in a query.

Following propositions can occur in the proposition object:

  1. $has

  2. $and

  3. $or

  4. $on

  5. $not

  6. $startsWith

  7. $gt

  8. $gte

  9. $lt

  10. $lte

  11. $defined

  12. $numeric 

  13. $knn

  14. Field proposition of form { "field" : PROPOSITION }

  15. The normal 'is proposition' containing a value primitive like "string", true, 4 or 4.3

As an additional note: the $numeric proposition no longer shows the numeric range.

You can consult the existing documentation to learn more about the proposition structure: https://aito.ai/docs/api/#language-text-operators

Back to developer docs

Locations

Kaivokatu 10 A, 8th floor

00100 Helsinki

Finland

See map

470 Ramona St.

Palo Alto

CA 94301, USA

See map

Contact

COVID-19 situation has driven us all to work from homes, please connect with us online. Stay safe & play with data!

Join our public Slack workspace

Follow us