Performance Guide

This guide covers Aito's performance characteristics and how to optimize for your use case.

Query Latency

Aito delivers fast query responses that scale well with data size.

Baseline Numbers

Measured on Intel Core i9-13900H, warm queries

What Affects Latency

Data characteristics:

• Row count (sub-linear scaling)
• Feature cardinality (more unique values = more computation)
• Text field length and complexity

Query characteristics:

• Number of fields in where clause
• Prediction target cardinality
• Use of basedOn with many attributes

System state:

• Warm vs cold cache (see Caching section)
• Concurrent write operations

Accuracy at Scale

Prediction accuracy remains stable as data grows:

Accuracy depends primarily on:

• Training data quality - representative examples
• Feature design - relevant attributes in schema
• Prediction difficulty - some fields are inherently harder to predict

Larger datasets generally help accuracy by providing more training examples.

Caching

Caching significantly affects query performance. Understanding it helps you optimize response times.

First Query vs Warm Query

Why cold queries are slower:

1. Indexes load from disk into memory
2. Inference caches populate
3. JVM warms up

Pre-warming

To avoid cold query latency in production, pre-warm after startup or data changes:

{
  "operations": [
    {"warm": [
      {"from": "invoices", "predict": "glCode"},
      {"from": "invoices", "predict": "processor"}
    ]}
  ]
}

Warm the queries your users actually run. See Warming Strategies for patterns.

Write Impact

Writes affect query performance. Understanding this helps you choose the right architecture.

What Happens During Writes

1. Insert/Update - Data added to write buffer
2. Buffer flush - Periodic flush to storage (every few seconds)
3. Cache invalidation - Some inference caches invalidated
4. Next query - May need to reload affected caches

Impact Severity

Solution: Write/Read Separation

For write-heavy workloads, separate your write and read tables:

1. Write to products_write
2. Periodically: optimize → warm → copy to products
3. Queries hit pre-warmed products table

{
  "operations": [
    {"optimize": "products_write"},
    {"warm": [{"from": "products_write", "predict": "category"}]},
    {"from": "products_write", "copy": "products", "replace": true}
  ]
}

See Warming Strategies for detailed patterns.

Memory Usage

Aito's memory footprint scales with data size:

JVM Configuration

For larger datasets, configure JVM heap:

# 1M rows
-Xmx1G -XX:+UseG1GC

# 10M+ rows
-Xmx4G -XX:+UseG1GC

Where these options go depends on how you run Aito:

When the same option is given more than once, the JVM applies the last one. The order is conf/application.ini, then JAVA_OPTS, then -J options on the command line, so a -J option overrides the others. In conf/application.ini, write options with or without the -J prefix (-Xmx4g or -J-Xmx4g).

Check the heap the server got in its startup log:

JVM max heap 4096 MB (set by -Xmx4g)

A line that says the JVM default means no option reached the JVM. Leave room outside the heap: Aito reads its data files through the operating system's page cache, so a heap that takes nearly all of the memory slows queries down instead of speeding them up.

Tuning for Your Use Case

Read-Heavy, Rare Writes

Simple warming after writes is sufficient:

// After data import
{"warm": [{"from": "products", "predict": "category"}]}

Frequent Writes

Use write/read separation:

• Write to shadow table
• Sync every 1-5 minutes
• Queries always hit warmed table

Real-Time Requirements

• Pre-warm common query patterns
• Keep hot data in smaller, optimized tables
• Consider query result caching at application layer

Large Scale (10M+ rows)

• Increase JVM heap (-Xmx4G or more; see JVM Configuration for where it goes)
• Use G1GC for better latency distribution
• Allow longer warmup time after restarts
• Consider partitioning by time or category

Benchmarking Tips

If measuring performance:

1. Warm up first - Run queries before measuring
2. Measure percentiles - P50, P95, P99 tell the real story
3. Isolate variables - Test reads separately from writes
4. Use realistic queries - Match production patterns

Related

• Warming Strategies - Managing cache and write impact
• Schema Design - Schema choices affecting performance