Win without breaking things
Pick the metrics that must not get worse. Kerf watches them for the whole test and rolls the change back if one crosses the line, before most users ever see it.
Three layers of protection
Guardrail metrics
Set a tolerance such as retention may not fall more than 1%. The check uses a one sided sequential test, so it can fire early.
Automatic rollback
When a guardrail fails with confidence, the variant is turned off and the owner is told in Slack with the numbers.
Sample size planner
Enter your baseline, traffic and the smallest effect worth shipping. Get users and days needed, before you launch.
Plan the test before you run it
Underpowered tests are the most common reason experiments say nothing. The planner shows how duration changes with the effect you care about, and warns you when the test would take longer than your release cycle.
- Power and significance set per workspace
- Accounts for CUPED variance reduction
- Handles ratio and revenue metrics
How a rollback happens
- 01
Watch
Guardrails are evaluated every 10 minutes against the exposed group and control.
- 02
Decide
If the lower bound of the interval crosses your tolerance, the check fails.
- 03
Act
Kerf reverts the flag to its last safe state and posts what it saw.
- 04
Review
The readout includes the guardrail trace and replay clips from the affected users.

“A guardrail caught a 4% retention drop on a launch everyone loved. We rolled it back in one click, before most users ever saw it.”
Guardrail questions
Can a guardrail be a system metric?
Yes. Pull error rate and latency from Datadog, Grafana or any SQL source alongside business metrics.
Will it roll back too eagerly?
Checks use sequential bounds and a minimum exposure, so noise at low traffic does not trigger a rollback.
Can I require a manual rollback?
Yes. Choose notify only, notify and pause, or full automatic rollback per guardrail.

