Performance Guide

This guide covers Aito's performance characteristics and how to optimize for your use case.

Query Latency

Aito delivers fast query responses that scale well with data size.

Baseline Numbers

Data Size	Mean Latency	Median Latency	Max Latency
10K rows	30-50ms	25-40ms	~200ms
100K rows	35-55ms	30-45ms	~300ms
1M rows	50-80ms	40-60ms	~500ms
16M rows	170ms	143ms	~800ms

Measured on Intel Core i9-13900H, warm queries

What Affects Latency

Data characteristics:

• Row count (sub-linear scaling)
• Feature cardinality (more unique values = more computation)
• Text field length and complexity

Query characteristics:

• Number of fields in where clause
• Prediction target cardinality
• Use of basedOn with many attributes

System state:

• Warm vs cold cache (see Caching section)
• Concurrent write operations

Accuracy at Scale

Prediction accuracy remains stable as data grows:

Data Size	Accuracy
10K rows	94-100%
100K rows	91-96%
16M rows	94.9%

Accuracy depends primarily on:

• Training data quality - representative examples
• Feature design - relevant attributes in schema
• Prediction difficulty - some fields are inherently harder to predict

Larger datasets generally help accuracy by providing more training examples.

Caching

Caching significantly affects query performance. Understanding it helps you optimize response times.

First Query vs Warm Query

State	Typical Latency
Cold (first query)	2-10x slower
Warm (subsequent)	Baseline

Why cold queries are slower:

1. Indexes load from disk into memory
2. Inference caches populate
3. JVM warms up

Pre-warming

To avoid cold query latency in production, pre-warm after startup or data changes:

{
  "operations": [
    {"warm": [
      {"from": "invoices", "predict": "glCode"},
      {"from": "invoices", "predict": "processor"}
    ]}
  ]
}

Warm the queries your users actually run. See Warming Strategies for patterns.

Write Impact

Writes affect query performance. Understanding this helps you choose the right architecture.

What Happens During Writes

1. Insert/Update - Data added to write buffer
2. Buffer flush - Periodic flush to storage (every few seconds)
3. Cache invalidation - Some inference caches invalidated
4. Next query - May need to reload affected caches

Impact Severity

Write Pattern	Query Impact
Occasional writes	Minimal - brief latency spike
Steady stream	Moderate - periodic spikes
Burst writes	Higher - consider write/read separation

Solution: Write/Read Separation

For write-heavy workloads, separate your write and read tables:

1. Write to products_write
2. Periodically: optimize → warm → copy to products
3. Queries hit pre-warmed products table

{
  "operations": [
    {"optimize": "products_write"},
    {"warm": [{"from": "products_write", "predict": "category"}]},
    {"from": "products_write", "copy": "products", "replace": true}
  ]
}

See Warming Strategies for detailed patterns.

Memory Usage

Aito's memory footprint scales with data size:

Data Size	Typical Heap
10K rows	100-150MB
100K rows	150-250MB
1M rows	300-500MB
10M+ rows	1.5-2GB+

JVM Configuration

For larger datasets, configure JVM heap:

# 1M rows
-Xmx1G -XX:+UseG1GC

# 10M+ rows
-Xmx4G -XX:+UseG1GC

Tuning for Your Use Case

Read-Heavy, Rare Writes

Simple warming after writes is sufficient:

// After data import
{"warm": [{"from": "products", "predict": "category"}]}

Frequent Writes

Use write/read separation:

• Write to shadow table
• Sync every 1-5 minutes
• Queries always hit warmed table

Real-Time Requirements

• Pre-warm common query patterns
• Keep hot data in smaller, optimized tables
• Consider query result caching at application layer

Large Scale (10M+ rows)

• Increase JVM heap (-Xmx4G or more)
• Use G1GC for better latency distribution
• Allow longer warmup time after restarts
• Consider partitioning by time or category

Benchmarking Tips

If measuring performance:

1. Warm up first - Run queries before measuring
2. Measure percentiles - P50, P95, P99 tell the real story
3. Isolate variables - Test reads separately from writes
4. Use realistic queries - Match production patterns

Related

• Warming Strategies - Managing cache and write impact
• Schema Design - Schema choices affecting performance