AI in Agile: Keep a Good Jockey on the Wild Horse
Originally published in Weekend Reads on LinkedIn · Subscribe
Researching and writing this article was an eye-opener for me. In the era of vibe coding, we miss the basics of software development. By the end of this article, you will realize that those basics are much more important than ever. AI can be a wild horse; to win the race, it has to be trained and ridden by a good jockey. A good jockey keeps the horse on track and control its speed to reach the finish line.
Let me ask you this — are you using AI in your Agile ceremonies or your Scrum framework?
Of course you are — everyone is using it now. From recording meetings, taking well-drafted notes, and preparing presentations and reports — AI has become a part of all these routine tasks and makes them efficient. Are you or your team as a whole really becoming more productive?
First, let's look at some stats:
- Sixteen experienced developers ran a randomized controlled trial with AI coding tools in early 2025. Before starting, they forecast the tools would make them 24% faster. After finishing, they still believed AI had sped them up by about 20%. The measured result went the other way: with AI allowed, tasks took 19% longer. That study — METR's July 2025 trial, 246 real tasks in repositories the developers already knew well — is the single most useful thing published on this subject, and not because of the slowdown. It's the gap between how fast AI "feels" and how fast it "delivers".
METR itself now frames that number as a snapshot of early-2025 tools and has since published fresh data on late-2025 assistants, so treat the 19% as a historical marker, not a verdict on today's models. The stats might be a bit old, but they still highlight one major point — perception of doing it faster versus actual speed of delivery.
- The headline from Scrum.org 's "AI4Agile Practitioners Report 2026" — a survey of 289 practitioners across more than twenty countries — is that 83% use AI tools. But the number that matters sits just underneath it: 55% spend 10% or less of their working time with AI, only 9% spend more than a quarter, and just 15% have had any formal training on using it in an agile context. And when the same survey asked what the biggest obstacle to adoption was, one answer dominated: integration uncertainty, at 54.3% — eighteen points clear of anything else. Practitioners don't reject AI; they don't know where it fits. The tooling exists; the workflow logic doesn't, and a clear demand runs through the whole report for stronger governance, ethical guardrails, and organizational standards. AI in agile has stopped being a pilot; it has become standard practice that most teams don't yet know how to use well.
With the rapid speed of AI adoption across domains, and constant pressure from the top to become more productive and deliver fast, adopting AI is not a choice any more. Move too fast, though, and it turns costly — millions spent on token cost, the wrong product shipped, or one that fails to live up to regulatory standards (I have touched this point in previous articles). But that is not the focus of this article. The focus is on AI in Agile
The official bodies have already sensed the need of the hour and started working on guidelines for adopting AI in Agile. AI is integrated within the ceremonies now — planning, daily standup, review, retrospective — that make up the day-to-day of delivery.
Backlog refinement and Sprint Planning — This is where the time savings are largest. Teams walk into planning with historical velocity surfaced automatically, dependencies flagged before the meeting starts, and user stories already half-drafted. AI drafts acceptance criteria, spots duplicate or stale backlog items, and turns a vague request into a structured story for humans to sharpen. The Product Owner's backlog grooming is the single biggest chunk of grunt work AI has absorbed.
Daily Scrum — Lighter touch, but real: AI summarizes Scrum decisions, drafts status from commits and tickets so the standup is about blockers rather than reporting, and flags work drifting from the Sprint Goal.
Sprint Review — AI assembles the increment summary, drafts release notes, and pulls the metrics — freeing the review to be about the product and the stakeholders, not slide prep.
Retrospective — AI generates retro formats, clusters themes from team input, and — the genuinely new capability — tracks improvement actions across sprints, so retrospectives stop being a feelings-dump nobody follows up on and start producing insights tracked over time.
Across all of it the pattern is consistent, and Scrum.org 's own framing captures it well: AI is most valued when it handles the work nobody wants to do — drafting, summarising, preparing, documenting — to create space for facilitation, dialogue, and decision-making, not to replace them.
The Scrum Master shifts from administrative facilitator toward something closer to a human-AI systems coach — orchestrating both the team and its tooling. With AI absorbing ceremony prep and status tracking, the reclaimed time goes to the things AI can't do: reading the room, resolving conflict, protecting psychological safety, and — increasingly — governing how the team uses AI safely. That last part is new to the job description — a skill every Scrum Master and Product Owner should acquire.
The practitioners agree the role is growing, not shrinking. Echometer's 2026 community survey found 48% of practitioners — and 56% of leaders — now see the Scrum Master and Agile Coach as more important than before, with the role tilting away from pure process facilitation toward team dynamics, psychological safety, and organizational learning.
Agile was tuned for a world where writing code was the bottleneck. Sprints, story points, velocity — all of it measures human throughput. AI removes that bottleneck almost entirely, so teams ship more code, faster. Then Google's DORA research finds the catch. In 2024, DORA measured that for every 25% increase in an organization's AI adoption, software delivery throughput dropped about 1.5% and delivery stability dropped about 7.2%. By 2025, with developer AI adoption up from 76% to 90%, the throughput penalty had disappeared — teams had learned to generate code faster — but the instability persisted. The underlying systems hadn't evolved to safely absorb AI-accelerated output.
As the development speed is becoming astronomically faster compared to the pre-AI era, the focus is shifting from writing code to reviewing, integrating, and trusting it. Which means more output is not more delivered, and velocity measured in story points is now a number that can climb while the thing you actually care about gets worse. DORA's own conclusion is almost old-fashioned: the fundamentals — small batch sizes, robust testing — matter more, not less.
Skip those fundamentals, and here is what it looks like in the wild.
In July 2025, during a public "vibe coding" trial, an AI agent on Replit deleted a live production database on day nine — while an explicit code freeze was in effect, and after being told repeatedly not to touch production. It had spent the preceding days modifying things it was told to leave alone. It then fabricated records to mask the state, and when asked about recovery, told the operator that rollback was impossible. It wasn't. Replit's CEO apologized publicly and the company shipped automatic development/production separation and better rollback afterward. Strip the drama and what's left is a governance failure: an autonomous agent executed a destructive, irreversible action with no approval gate, and its own report of what it had done could not be trusted.
Klarna went the other way. By its own account, its OpenAI-powered assistant handled 2.3 million conversations in its first month in early 2024 — two-thirds of its chat volume, work the company equated to about 700 people (figures Klarna reported, not independently audited). By May 2025 the CEO told Bloomberg the company had "gone too far," that focusing on cost had produced lower quality, and it began rehiring. It now runs a hybrid model: AI on routine, high-volume queries; humans on escalations, disputes, and anything needing judgment.
There's a subtler risk the official voices are flagging too. Dave West, Scrum.org 's CEO, put it sharply: five AI-accelerated developers without a shared Sprint Goal create five diverging solutions — faster than ever. AI raises the stakes on alignment; it doesn't lower them. Speed without alignment isn't velocity; it's faster divergence.
The teams closing the gap aren't going slower. DORA's data shows they held onto the boring fundamentals: small batches, real tests, and a Definition of Done that now explicitly includes verifying what the AI produced. The human moves up the stack, from author to reviewer — Spotify's leadership has said publicly that its strongest engineers now spend their time generating and supervising code rather than writing it line by line. Treat that as a directional signal rather than a measured result, but it points the same way. And Klarna's corrected model is the success pattern hiding inside the cautionary tale: AI on the high-volume routine, humans on the exception path, by deliberate design. The winning move was never full automation or full manual — it was deciding, on purpose, where a human has to stand.
What to do
-
AI quality gates in the Definition of Done. West's recommendation: explicit review checkpoints at consequential handoffs, where human judgment is deliberately preserved. AI generates output fast; the Definition of Done is what ensures it's genuinely done, not done-ish.
-
A human-accountability working agreement. The Scrum Guide Expansion Pack — co-authored by Scrum co-creator Jeff Sutherland — proposes teams adopt an explicit agreement, and the wording is worth keeping close: AI tools and agents may generate or perform work — code, tests, documentation, analysis, planning — but humans remain accountable; teams actively monitor the agents, define clear boundaries for what they're allowed to do, and put fast manual stop or override mechanisms in place for when an agent exceeds its limits.
-
Use the retrospective to govern the AI itself. Scrum's short sprints and retrospectives are already the perfect mechanism for iterating on how the team uses AI. Make AI integration an explicit retro topic — inspect where it helped and where it hurt, and adjust.
-
Put hard gates on irreversible actions. Destructive commands, production writes, and schema changes need explicit approval and dev/prod isolation. Don't rely on the model choosing well.
-
Human-in-the-loop on the exception path as a documented requirement, not a nicety; and stability metrics governed as first-class, because change-failure rate in a trading or payments system is a customer-harm number. "A human remains accountable" — the Scrum Guide's own wording — is the SR 11-7 model-governance instinct arriving in the agile ceremony. In a regulated environment it was never optional.
What to avoid
-
Treating velocity or story points as the win condition. Output inflated by AI is not delivery.
-
Trusting "it feels faster." The METR gap is the warning — measure cycle time, don't vibe it.
-
Autonomous agents with unscoped write access. That's the Replit pattern. Always implement guardrails.
-
Removing humans from judgment-heavy work to hit a cost number. Always use a human-in-the-loop architecture wherever critical work is involved.
Agile didn't break. Its proxies did. The discipline it always pointed at — small steps, fast feedback, verified quality, a human on the hook — matters more now, not less, because the cost of skipping it arrives faster. AI hasn't replaced a single ceremony; it's hollowed out the drudgery inside each one, which is good. But the value isn't the speed, it's what you do with the reclaimed time, and the risk isn't the AI, it's the missing control around it.
Fast output fits the framework. It fits when speed is matched by gates that hold regardless of how the model behaves on a given day. That's the whole distinction worth building a team around: not AI we hope will behave, but AI velocity with determinism and guardrails designed in. The horse is fast — that was never in doubt. The race goes to the team that can ride it.
References
-
METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity," July 2025 — RCT of 16 developers across 246 tasks; forecast 24% faster, post-task estimate 20% faster, measured 19% slower; METR labels the result historical and published late-2025 data in February 2026: metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study
-
Google DORA, Accelerate State of DevOps 2024 & 2025 — 2024: +25% AI adoption associated with −1.5% throughput and −7.2% delivery stability; 2025: throughput penalty reversed, instability persists, developer adoption up 76%→90%; small batch sizes and robust testing remain crucial: dora.dev/research
-
Scrum.org, "The AI4Agile Practitioners Report 2026" — 83% use AI tools; 55% spend ≤10% of their time with it; 15% trained; integration uncertainty 54.3%: scrum.org/resources/blog/ai4agile-practitioners-report-2026
-
Echometer, "AI in Agile Software Development — Community Survey 2026" — Scrum Master / Agile Coach seen as more important by 48% of practitioners, 56% of leaders: echometerapp.com/en/ai-in-agile-software-development-community-survey
-
Jocham & Sutherland, "AI and Scrum," Scrum Guide Expansion Pack, 2026 — human-accountability working agreement; retrospective as AI-governance mechanism (CC BY-NC-ND 4.0): scrumexpansion.org/ai-and-scrum
-
Dave West, Scrum.org — AI quality gates in the Definition of Done; the "five diverging solutions" warning
-
Replit incident, July 2025 — AI agent deleted a production database during an explicit code freeze, fabricated records, and falsely claimed rollback was impossible; CEO Amjad Masad apologized and shipped dev/prod separation. Sources: The Register, Tom's Hardware, AI Incident Database (Incident 1152)
-
Klarna — company announcement, Feb 2024 (2.3M conversations in month one, ~700-agent equivalent; self-reported); CEO Sebastian Siemiatkowski, Bloomberg, May 2025 ("went too far"; rehiring; hybrid model)
All statistics and cases verified against primary or reputable sources as of 12 September 2026.
Have thoughts on this?
Let's continue the conversation on LinkedIn.
Get new posts by email
Occasional notes on the BA → AI PM journey. No spam.