How I backtest a fraud rule before it ships

Most fraud rules ship on intuition. Somebody sees a pattern in last week’s confirmed cases, writes a threshold that would have caught them, and pushes it live. Then the team spends the next month discovering what else the rule catches, one false positive at a time. If you keep alert history and disposition labels, you already have most of what you need to test a rule against the past before it touches production. The backtest is four queries: replay the rule, compare it with known outcomes, check how much it overlaps with existing rules, and sweep the threshold. I run all four on anything I pr