Hearthly learned why variants win, not just which one
A home goods marketplace used AI readouts and linked replays to turn test results into decisions the whole team could follow.

Results nobody read
Hearthly runs a marketplace for independent furniture makers. The growth team ran plenty of tests, and the results lived in slide decks that people skimmed. A test would end with a p-value and a shrug, and the next test would start from scratch.
A readout that explains itself
Kerf's AI readout writes a plain language summary of every test. It states the primary result with its interval, lists the segments where the effect differed, checks guardrails and sample ratio, and says whether the data supports shipping, stopping or running longer.
- It cites the exact metric definitions and the exposure window.
- It links replay clips of users in each variant.
- It flags results that look too good, such as a lift driven by one large customer.
Replays change the conversation
On a test of a new product gallery, the numbers were flat. The readout linked replays showing buyers on small phones zooming into photos and missing the add to cart button, which was pushed below the fold. A one line layout fix turned a flat result into a win.
Results
Readouts that once took an analyst two days now arrive when the test ends. In 94% of tests the team leaves the review with a clear decision. Tests that end flat are logged with a reason, so the same idea is not tried twice by accident.
Leadership reads the readouts directly. The weekly growth review went from 90 minutes to 30.

“The AI readouts surprised me most. Kerf explains why a variant won, with replay clips, before I've even opened a dashboard.”
More customer stories

Northwind cut checkout drop-off by 18%
A retailer with 2,400 employees moved every checkout change behind a flag and a test, and found the fix was a smaller form, not a redesign.

Halcyon ships weekly instead of monthly
A health scheduling platform replaced a manual release process with gradual rollouts and automatic rollback.

Brightpath replaced three tools with one
An education platform consolidated flags, experiments and session replay, and the arguments about whose numbers were right stopped.