Alot Labs
Menu
Work
Deployments Lab Live Writing
Firm
About Capabilities Services Method Pricing Careers The commitment Compare the alternatives Book the briefing
← All deployments
Case study · Deployment 02

AutoTest

A one-line prompt becomes a self-healing browser test suite that monitors your application on its own schedule. Built and operated by us, end to end.

Playwright execution Multi-model consensus Self-healing selectors Autonomous monitoring
TypeSelf-healing QA platform StatusIn production — run at our own expense StackPlaywright · cross-model consensus Runs inSelf-hosted today · your VPC on engagement
The problem

End-to-end test suites rot. Every UI change breaks selectors, every broken selector needs an engineer, and maintenance quietly becomes the biggest line item in QA — until the team stops trusting red builds. AutoTest removes the maintenance instead. We ran it against real applications at our own expense before writing a word about it here.

1
line of plain English is all it takes to generate a runnable browser test
2
frontier models must independently agree before any assertion passes or fails
0
model calls on a healthy run — verified selectors replay from cache, deterministically
24/7
autonomous monitoring between releases — no human pressing run
What it does
01
Generates

A natural-language brief — “test the login flow” — becomes a runnable Playwright test. No test DSL to learn, no selector spelunking.

02
Executes

Real browser, real application. Assertions are judged by cross-model consensus — two frontier models must agree before a state is called passing or failing.

03
Self-heals

When the DOM shifts, broken selectors are re-derived, the test is patched, and the verified selector is cached for every run after. Healing repairs the path to an assertion — never the assertion itself.

04
Monitors

A background scheduler runs suites on their own cadence between releases, ranks tests by failure risk, and writes a plain-language summary of what changed overnight.

05
Remembers

Every verified selector, anti-pattern, and confirmed step persists into project memory. The next test generated inherits everything the platform has already learned about your app.

Architecture
One-line prompt “test the login flow” Generation Claude → executable steps Scheduler 24/7 · risk-ranked runs Selector cache verified · replays free Playwright execution real browser, real app Consensus gate 2 models must agree selector broke expectation failed Self-heal repairs the path, never the assertion verified fix cached Defect filed full trace evidence — never healed away Project memory selectors · anti-patterns · confirmed steps the next test inherits everything learned

Stack: TypeScript throughout — React + Vite front end, Express API, Postgres via a typed ORM, Playwright execution, Claude as the primary model with a secondary frontier model as assertion arbiter, Redis for the verified-selector cache.

What broke, and what we changed
One judge flaked, so we required two

One model judging “did the page reach the right state?” produced false positives no team would tolerate in CI. Every assertion now needs 2 independent verdicts: a second frontier model judges the same evidence, and a disagreement goes to arbitration — never straight to your build status.

Retry-until-green is how a testing tool becomes a lying tool

So we drew a structural line: healing may repair how a test reaches an assertion (selectors, navigation), and exactly zero of what the assertion expects. A failed expectation files as a defect with full trace evidence, 100% of the time — which is why the diagram has two separate exits from a failed run.

The model speaks at exactly 2 moments in a test’s life

Generation, and breakage. Every healthy run in between replays cached, verified selectors deterministically — 0 model calls, so the steady state is fast, reproducible, and effectively free. Model spend concentrates where judgment is needed; everything else is replay.

The demo

Our demo runs the full platform against a real open-source CRM: generated tests, run history with a live self-heal recovery, learned project memory, and autonomous monitoring cycles. Nothing is mocked — every number on screen comes out of the database.

In production

Suites generate from one-line prompts, verified selectors persist across runs, and the platform monitors between releases instead of waiting for a human to press run. We operated it ourselves, on our own applications, at our own expense.

What we would deploy for you

AutoTest lands in your cloud, wired into your CI and pointed at the application blocking your release train, with the consensus gate tuned to your team's false-positive tolerance. Your application and data never leave your environment.

Where this is going

The ticket that requests a feature, the doc that describes it, and the test that verifies it are three copies of the same intent, drifting apart from day one. Our next platform, now in the lab, collapses them into one object: prose that is at once the requirement, the documentation, and the executable test. AI proposes; humans keep final judgment.

If staging is where your releases slow down, ask about it in the briefing.

Related deployments
Book the briefing Send the workflow