Vaishali Desarda - Quality, AI, and the thinking in between!
July 19, 2026
⌚ : 6 min
                                     Click to find relevant articles:

Built on Scar Tissue - Part 3: The Story Was Already Wrong

The defect that nobody wrote, but everyone shipped

"The single biggest problem in communication is the illusion that it has taken place."

- George Bernard Shaw

Here's a Jira story that would sail through grooming:

Story: Send USDT to another wallet. As a crypto user, I want to transfer USDT from my wallet to another wallet.

User enters recipient address. Enters amount. Clicks Send. System validates balance. Creates transaction. Shows success.

Everyone nods. Three story points. Development starts.

Here’s what actually happens. A user sends 10,000 USDT. The network’s slow. The button shows a spinner. She clicks Send again. Then again.

Three valid requests. All pass balance validation - because the second and third checks run before the first balance updates. Three real blockchain transactions. 30,000 USDT sent instead of 10,000. Blockchain doesn’t have undo. Somebody is holding that loss.

During refinement, somebody should have asked the question: what happens if the user clicks Send more than once?

Well, nobody asked. Not because the team was careless. Six acceptance criteria, all looked complete. The question just wasn’t in the room.

Six AC

Same gap, no human in the loop

Now take the same gap and remove the human entirely.

This specific failure hasn’t made it to a retro room yet - agentic financial tools are still too new for that. But anyone who’s watched this gap repeat before can see where it’s heading.

OpenClaw, the self-hosted agent that went viral late last year, lets users move money, interact with DeFi protocols, and manage wallets through natural language -make this less of a thought experiment and more of a question worth asking before it happens. OpenClaw can move money, interact with DeFi protocols, and manage wallets. Exactly the kind of tool where this failure mode becomes someone’s real retro story. Imagine a user says: “Send 1000 USDT to Mum.” The agent finds Mum’s wallet, validates the balance, and calls a transfer tool.

User → LLM → Transfer Tool → Blockchain

The transfer goes through on-chain. Then something ordinary happens: a timeout, a dropped connection, the agent process restarting. The user never sees a success message. They see: something went wrong.

So they do the obvious thing. They try again.

The agent reasons through the request fresh. It has no memory of the fact that it already succeeded thirty seconds ago. It calls the transfer tool again. The blockchain executes a second, completely valid transfer. The user now believes they sent 1000 USDT. They actually sent 2000.

Notice what didn’t go wrong here. The LLM reasoned correctly. The tool executed correctly. The blockchain recorded everything accurately. Every individual component did its job. The failure lives entirely in a question nobody had written down anywhere in the system: can the agent tell the difference between retrying something that failed and repeating something that already succeeded?

The agent is doing its job perfectly.

The job was never specified correctly.

A year ago, this would have sounded like an edge case. Now it’s just what happens when a system takes real-world action - books something, sends something, charges something - and gets interrupted between calling a tool and hearing back from it.

One stage earlier than Part 1 and Part 2

This is the gap Part 1 and Part 2 both pointed at, but one stage earlier.

Part 1 showed how AI-generated test suites look complete while missing the thinking. Part 2 showed how Release Risk Intelligence catches what generation misses by asking who else the system touches. Both operate after the code exists.

This operates before it.

The question isn’t “does the test suite cover the risk?” It’s “does the story even know the risk exists?”

That’s a different category of intervention. Not a test generator. Not a code reviewer. Not a Jira summarizer. Something that reads a story before a sprint starts and asks one question nobody wrote down.

RRI reviews test suites - catches what the tests missed. This one reviews stories - prevents the gap from reaching the tests at all. Same philosophy, different stage. Both built from the same scar tissue.

That’s been the arc: generation makes you feel covered. Review catches what generation missed. One stage before both - the story itself was already wrong.

The question the reviewer surfaces

Give the original story to the reviewer. Here’s the question it identifies as missing - not a test case, not a code review, but the gap that would have caught this before build:

If a user submits the same transfer request more than once - whether through impatience, network retry, or an AI agent re-invoking a tool after a connection timeout - does the system guarantee that only one transaction is created? If not, what prevents the duplicate from succeeding?

The story happens to be about crypto. The question doesn’t. I’ve watched the same question surface in health monitoring, food delivery, payment systems, and now agentic AI. Different technology. Different teams. Different decade. Same missing assumption. That’s what scar tissue looks like.

Same scar tissue. Different actor. The OpenClaw scenario above is that question one level deeper - a human clicking twice versus an agent retrying twice, with no instinct to hesitate. I haven’t run that one through the reviewer yet. That’s Part 4.

That question, asked during grooming, would have changed the story before a line of code was written.

An engineer who’s been burned by double submissions once might add idempotency to that specific endpoint. An engineer who’s been burned four times doesn’t wait for the endpoint. They check every story that touches an irreversible operation.

Ask a coding assistant to build the transfer flow from that original story, and it builds exactly what’s written - validate, transfer, confirm. Nothing in the story called for an idempotency check. The model didn’t miss it. The story never asked for it.

What experience actually teaches

The acceptance criteria aren't the insight. The question is.

The criteria are what you write after you know what to ask. Experience doesn't give you better answers. It teaches you where to become suspicious.

In the teams I’ve watched get this right, somebody asks one question before grooming closes, and asks it of every story that touches money, state, or anything irreversible: what happens if this exact request arrives twice - from a slow human, a flaky network, or now, a retrying agent?

Most teams ask what should happen. Good teams ask what could happen twice. Recently I’ve noticed another question starting to appear. Not because the systems changed. Because the actor did. The thing retrying isn’t a person anymore. It’s another system. And systems don’t hesitate.

Thirty seconds. In a meeting most people are half-listening to. That single question is the difference between a clean three-point story and a retro nobody enjoys.

What’s next

The agent is doing its job perfectly.

The job was never specified correctly.

Part 4 is where that sentence stops being philosophy and becomes a product: The Question Before the Code - a tool that reads a story before the sprint starts and hands back the question nobody realised they needed to answer. Not tests. Not coverage. Just the question.

Coming soon.

If you’ve shipped something that did exactly what the story said, and exactly what nobody meant - I’d like to hear where the gap was.


Connect / Follow me

Quality, AI, and the thinking in between.