# The Death of the Flaky Test: Why I Stopped Writing Scripts and Started Architecting Agents

It’s 2026. If you are still manually updating CSS selectors because a div moved three pixels to the left, you are doing it wrong.

For the last decade, we’ve been stuck in a loop of "write, break, fix, repeat." We called it "Automation," but it felt a lot more like babysitting. We built fragile Rube Goldberg machines that screamed every time a developer changed a class name.

Check out the project here: [https://github.com/iddimov/llm-playwright](https://github.com/iddimov/llm-playwright)

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1770938081160/127e5ab5-8707-4d78-937b-7d1842b87630.png align="center")

I recently spent some time digging into the **LLM-Playwright Automation Framework**, an open-source project that finally feels like the exit ramp from this maintenance hell. It’s not just another wrapper around Selenium. It’s a glimpse into the actual future of Quality Engineering - where we stop writing scripts and start architecting **Agentic Systems**.

Here is the gap analysis of why the old way is dying, and a look at the tech stack - specifically the **Planner-Generator-Healer** pattern - that is replacing it.

## The "Blindness" Problem

The fundamental flaw of traditional automation (Selenium, Cypress, and yes, vanilla Playwright) is that it is **context-blind**. A script doesn't know *what* a login button is; it only knows that it’s looking for `#btn-primary-login`. If that ID changes, the script fails. It has no eyes, no intuition, and no ability to adapt.

We tried to fix this with "Self-Healing" tools in 2024, but most were just glorified try-catch blocks with a dictionary of backup selectors.

The shift to **Agentic Engineering** changes the primitive. We aren't giving the computer a list of steps anymore. We are giving it **sight** via the **Model Context Protocol (MCP)** and a **brain** via reasoning models like OpenAI’s o1 or DeepSeek’s R1.

## The Nervous System: MCP & `mcp-use`

This project is built on a stack that I think will be the standard for 2026: **Node.js**, **TypeScript**, **LangChain**, and most importantly, **MCP**.

If you haven't touched MCP (Model Context Protocol) yet, think of it as the USB-C for AI. Before MCP, connecting an LLM to a browser was a mess of ad-hoc function calling and prompt injection. You had to paste the HTML into the prompt and pray the token limit didn't cut you off.

With MCP, the browser becomes a **Server**. It exposes its accessibility tree, network logs, and console as structured **Resources**. The Agent is the **Client**.

This framework uses a library called `mcp-use` to bridge the gap. It’s a unified client for Node.js that handles the messy handshake between your LLM and the tool.

Here is why this matters: Security and Stability. Instead of giving an agent raw `eval()` access to your browser (terrifying), `mcp-use` creates a strict contract. The agent can only "click," "fill," or "navigate" because those are the only tools the MCP Server exposes.

```typescript
// A glimpse of how clean the mcp-use integration is
import { MCPAgent, MCPClient } from 'mcp-use';

const client = MCPClient.fromDict({
  mcpServers: {
    playwright: {
      command: 'npx',
      args: ['@playwright/mcp-server'] 
    }
  }
});
```

This simple setup allows the agent to "see" the page the way a human does - by semantic meaning ("the button that says 'Checkout'"), not by arbitrary DOM structure.

## The Trinity: Architect, Developer, Janitor

The brilliance of this framework isn't just the tools; it's the **Multi-Agent Architecture**. It breaks the testing lifecycle into three distinct personas. This is the "Mixture of Experts" pattern applied to QA.

### 1\. The Planner (The Architect)

*   **Model:** High-reasoning (OpenAI o1 or DeepSeek R1).
    
*   **Job:** Strategy.
    
*   **Input:** "Test the checkout flow."
    

The Planner doesn't write code. It explores. It browses the app, clicks around, and maps the territory. It handles the cognitive load that used to burn us out: finding the edge cases. It outputs a structured Markdown plan (`specs/`[`coverage.plan.md`](http://coverage.plan.md)) that details *what* needs to be tested, covering happy paths and negative scenarios.

### 2\. The Generator (The Developer)

*   **Model:** High-coding capability (GPT-4o or DeepSeek V4).
    
*   **Job:** Execution.
    
*   **Input:** The Planner's Markdown.
    

This agent takes the plan and writes the Playwright code. But it doesn't just spit out spaghetti code. It adheres to the **Page Object Model (POM)**. It creates strictly typed TypeScript files in `pages/` and `tests/`. It treats test code as production code.

### 3\. The Healer (The Maintainer)

*   **Model:** Fast & Cheap (Llama 3 or DeepSeek V3).
    
*   **Job:** Resilience.
    
*   **Input:** A failed test report.
    

This is the killer feature. When a test fails, the Healer wakes up. It reads the Playwright trace, looks at the error (e.g., "Element not found"), and looks at the *current* DOM via MCP.

It realizes, "Oh, the dev changed the button ID from `#submit` to `#complete-order`, but it's still the same button." It updates the selector in the code, runs the test again, and if it passes, it commits the fix. **Zero human intervention.**

## The Economics of Autonomy

"But isn't this expensive?"

In 2023, maybe. In 2026, no.

The cost of running a Planner agent to generate a suite might be $2.00 in tokens (depends on the model used 😏). The cost of an SDET spending 4 hours writing that same suite is slightly more…

Furthermore, we have **Context Compaction**. We don't feed the entire history to every agent. The Generator only sees the Plan, not the Planner's internal monologue. We use **Prompt Caching** to cache the system instructions (the "How to write Playwright" rules), so we only pay for the new logic.

And let's talk about **DeepSeek**. The framework supports it natively. Using DeepSeek V4 for the heavy code generation cuts costs by nearly an order of magnitude compared to GPT-5 class models, without losing accuracy on syntax.

## The Verdict

This isn't just a cool repo; it's a new operating model.

The **LLM-Playwright Automation Framework** demonstrates that we are moving toward a world where humans define the *Intent* ("Ensure the user can pay"), and the AI handles the *Implementation* (Selectors, waits, retries).

If you are an engineer, your job is shifting. You are no longer a script writer. You are an Agent Architect. You define the constraints, the tools, and the goals. The agents do the clicking.

Stop fixing flaky tests. Let the robots do it.