Rule Precision Replay

Every promoted rule replayed against the tenant's invoices: how many invoices it matches, how often it disagrees with the posted value, and its precision against ground truth.

Rule Precision Replay: screenshot from the Accounting demo
🧾 Accounting_relate_predict_searchAccountingERP
Production anchorRule Performance in the accounting demo: the rules in force, mined with _relate and promoted by a person, are replayed over the tenant's invoices and scored for matches, disagreements, and precision; Demote takes a rule out of routing.

The problem

Rules are trusted because they are explicit, and then nobody checks them. A rule written when a vendor always coded to one account keeps firing after the vendor's mix changes, and the errors look like rule output, so nobody questions them.

A rule deserves the same scrutiny as a prediction: how often does it fire, and how often is it right?

How it works

The rules are mined from the tenant's own data with _relate, and only the ones a person has promoted route invoices. Each promoted rule is then replayed over a sample of the tenant's invoices: where its condition matches, the rule's value is compared with the value that was actually posted. The result per rule is its match count, its disagreements, and its precision, next to who promoted it and when.

A rule whose precision drops is a candidate for review, the same way a prediction below its confidence threshold goes to a human. Demote takes it out of routing and records the event, so the audit trail keeps both the promotion and the demotion.

For the full architecture, see the technology overview. For the broader narrative across multiple use cases, read The Predictive Application.

See it live

This use case runs in the 🧾 Accounting demo today. Click through to the live application and inspect the queries that produce the result.

Open the live demo →