Warming Strategies

This guide explains how to use warming operations to keep your Aito database responsive after write operations.

Why Warming Matters

After write operations (insert, update, delete), the first query against modified data can experience significant latency spikes—sometimes 10 seconds or more. This happens because:

  1. Index reloading: Indexes need to be loaded from disk into memory
  2. Inference cache population: ML inference requires numerous internal caches to achieve fast response times

No disk operations occur during query execution itself, but the initial loading and cache population can cause substantial delays. The warm operation allows you to pre-execute queries to populate these caches before real users hit them.

Note: The severity of these latency spikes depends on your data size and feature space complexity. Small tables with simple schemas may not need warming at all, while large tables with many features can experience significant first-query delays. Evaluate your specific use case before implementing warming strategies.

Single-Table Pattern

The simplest pattern: warm your table immediately after modifications.

{
  "operations": [
    {"into": "products", "insert": {"id": 123, "name": "New Product", "category": "electronics"}},
    {"optimize": "products"},
    {"warm": [
      {"from": "products", "predict": "category"},
      {"from": "products", "recommend": "related_products"}
    ]}
  ]
}

Trade-offs:

  • Simple to implement
  • Write latency includes warming time
  • Best for low-write-frequency scenarios

Write/Read Separation Pattern

For high-frequency writes, separate your write and read tables:

  1. Write to a dedicated write table
  2. Periodically optimize, warm, and copy to the read table
  3. Queries always hit the pre-warmed read table
{
  "operations": [
    {"optimize": "products_write"},
    {"warm": [
      {"from": "products_write", "predict": "category"},
      {"from": "products_write", "recommend": "related_products"},
      {"from": "products_write", "where": {"popular": true}, "predict": "price_range"}
    ]},
    {"from": "products_write", "copy": "products", "replace": true}
  ]
}

Trade-offs:

  • Writes are fast (no warming delay)
  • Read queries always hit warmed data
  • Data staleness window between syncs
  • Requires periodic sync job

Example: E-commerce Product Updates

// Step 1: Write updates to the write table (fast)
{
  "into": "products_write",
  "insert": {"id": 456, "name": "Premium Widget", "category": "accessories"}
}

// Step 2: Periodic sync job (e.g., every 5 minutes)
{
  "operations": [
    {"optimize": "products_write"},
    {"warm": [
      {"from": "products_write", "predict": "category"},
      {"from": "products_write", "recommend": "frequently_bought_together"}
    ]},
    {"from": "products_write", "copy": "products", "replace": true}
  ]
}

When to Use Optimize

The optimize operation in the examples above is optional. It merges log-structured storage segments into a single file, which can significantly improve query speed. However, optimize itself is a slow operation.

Consider using optimize when:

  • You have accumulated many small writes
  • Query performance is degrading over time
  • You're doing a periodic sync in the write/read separation pattern

Skip optimize when:

  • You need fast write-then-query turnaround
  • The table has few segments (recent writes only)
  • Write latency is more important than query latency

Without optimize, warming still works—it just warms the current table state.

Choosing Queries to Warm

Not all queries benefit equally from warming:

Good candidates for warming:

  • predict queries - These involve model computation
  • recommend queries - Similar to predict
  • similarity queries - Vector similarity search
  • Complex where clauses with multiple conditions

May not need warming:

  • Simple search queries - Already fast
  • Queries with limit: 1 - Minimal computation
  • Aggregate queries on small result sets

Performance Considerations

  1. Warm the queries your users actually run - Profile your production query patterns and warm those specific queries.

  2. Don't over-warm - Warming too many queries increases sync time and may warm caches that get evicted before use.

  3. Balance freshness vs latency - Shorter sync intervals mean fresher data but more frequent warming overhead.

  4. Monitor first-query latency - Track your P99 latency to verify warming is effective.