May 28, 2026 · ← Notes
The promise of AI automation is that you set it up once and it runs. The reality is that AI automations fail differently than the automations we're used to — and the difference is what makes them dangerous.
A traditional automation, when it breaks, usually breaks loudly. The Zap stops firing. The script throws an error. The integration shows a red banner. You find out because something stops happening.
AI automations don't do you that courtesy. They keep running. They keep producing output. The output just quietly stops being correct — and because there's no error, nothing alerts you. You find out weeks later, when a customer points out something strange, or when you happen to look closely at what your system has actually been doing.
I've built enough of these to have a healthy fear of the specific ways they go wrong. Here are the failure modes worth knowing about — because once you've seen them, you start designing around them.
You build a workflow where AI drafts something — a support reply, a summary, a categorization. You test it on ten examples. It nails all ten. You ship it. What you didn't test: the eleventh example, which is slightly outside the distribution of your test cases.
The AI doesn't error on these. It produces something plausible-looking and wrong. Traditional code handles the cases you wrote it to handle and visibly fails on the rest. AI handles everything — including the cases it has no business handling — and fails invisibly. The output is always confident. The confidence is not correlated with correctness.
The mitigation isn't more testing up front. It's sampling the output continuously after launch.
Say you have AI extract structured data from messy inputs — pulling fields out of emails, parsing documents, categorizing records. Most of the time it's right. Occasionally it invents a value. Not a blank, not an error — a confident, plausible, wrong value.
Now that value flows into your CRM. It triggers a downstream automation. It gets included in a report. It informs a decision. By the time anyone notices, the bad data isn't a single wrong cell — it's propagated through five systems and shaped three decisions.
If an AI step writes data that other steps depend on, you need a validation layer between them.
You wrote a prompt. You tuned it carefully. It produced exactly the output format your downstream automation expected. Then the model provider shipped an update. The new version is "better" by their benchmarks — and it formats its output slightly differently. Now your downstream parsing silently mangles every result.
This is a genuinely new category of fragility. Traditional software dependencies are versioned and pinned. Many AI integrations run against a model endpoint that can change underneath you.
The mitigation is to pin model versions where the provider allows it, and to build output validation that fails loudly when the format drifts.
AI workflows usually chain several tools. The dangerous version isn't when a connection breaks completely. It's when it half-works. The webhook fires but the payload is malformed. The API returns a 200 but the body is empty. The AI step times out and returns a partial response that happens to be valid enough to pass to the next step.
Each of these produces a result. None of them produces an error. The automation reports success. The success is a lie.
You manage this by checking outputs at each handoff, not just at the end.
AI automations often have usage-based pricing. A workflow that costs pennies per run is fine until something changes the run volume. A retry loop that should fire occasionally starts firing constantly. You don't get an error. You get an invoice.
The mitigation is boring but essential: usage alerts on anything with consumption-based pricing, and a periodic look at actual spend versus expected spend.
Every one of these failure modes shares a property: the system keeps reporting success while the success becomes false. That's the thing to internalize about AI automations. They don't fail like software. They fail like a confident employee who's quietly stopped doing the job correctly but keeps showing up and telling you everything's fine.
The design principle that follows is simple to state and easy to skip: never trust an AI step's output without a way to verify it. Sample the results. Validate the structure. Check the data before it flows downstream. Watch the costs. The watching is the job.