BlogProduct engineering

AI coding agents: what changes for software teams

AI coding agents take a task, read the codebase, edit several files, run the tests and open a pull request for a person to review. That moves the bottleneck from writing code to specifying work and checking it. Teams that get value from agents invest in tests, small tasks, written conventions and tight permissions. Teams that skip those get more code and more incidents.

From autocomplete to agents

Coding assistance has gone through three stages. First came completion, which suggests the next few lines as you type. Then came chat, where you paste code and ask questions. The agent is the third stage: you describe an outcome, and the tool plans, edits files, runs commands, reads the results and tries again until the tests pass or it gets stuck.

The main products now all work this way. We checked their documentation while writing this.

  • Claude Code runs in the terminal, in IDEs, in a desktop app and on the web. It edits files, runs commands, creates commits and pull requests, and can run several agents in parallel or on a schedule.
  • GitHub Copilot’s cloud agent researches a repository, plans a change and implements it in the background in a cloud environment. Sessions that run on GitHub Actions are capped at 59 minutes, and each task produces one pull request.
  • OpenAI Codex comes as a command-line tool, an IDE extension and Codex Cloud, where each task gets its own workspace and can end in a commit or a pull request.
  • Cursor offers Cloud Agents, formerly called Background Agents, which work on a branch in isolated virtual machines and can be started from the web, Slack, GitHub or Linear.

They also share conventions. AGENTS.md, a plain Markdown file of build steps, test commands and house rules, is now stewarded by the Agentic AI Foundation under the Linux Foundation and is read by most of these tools.

Two modes matter in practice. In the interactive mode a developer steers the agent in their own checkout and sees every step. In the background mode a task is handed over from a ticket or a chat message and comes back as a pull request. The second is where the working habits of a team have to change.

What do agents do well and badly?

This is our experience on real codebases, not a benchmark.

Usually works Usually goes wrong
A well-specified change in code that has tests Vague requirements, where the agent picks an interpretation and builds it confidently
Mechanical work: migrations, dependency upgrades, lint fixes, renames across many files Changes that depend on things outside the repository, such as another team’s service or a production data quirk
Writing tests for existing code Making a failing test pass by weakening the test
Reproducing a bug from a stack trace and proposing a fix Performance and concurrency problems that only appear under load
Explaining unfamiliar code and tracing how a request flows Large refactors in code with no tests to check against
First drafts of scripts, internal tools and glue code Reusing what exists: agents often write a new helper beside three old ones

The pattern is consistent. An agent is strong when the task is bounded and the feedback is quick and reliable, and weak when success depends on knowledge that is in people’s heads. It will also keep going when it should stop and ask. Long tasks drift, so shorter ones with a clear finish line give better results.

How review, testing and architecture change

Review becomes the bottleneck. An agent can open more pull requests in a morning than a team can read in a day. If review capacity stays the same, either the queue grows or the reviews get shallow, and shallow review of machine-written code is how defects reach production. Keep pull requests small. Require the agent to state what it changed, what it ran and what it could not verify. Read the test changes first. An automated review pass is useful as a first filter and is not a replacement for a person who knows the system.

Tests become the specification. The agent treats the test suite as its definition of done. A fast, deterministic suite gives it a reliable signal. A slow or flaky one teaches it to retry until something goes green. Money spent on test speed and coverage now pays back twice, once for people and once for agents.

Architecture has to be legible. Clear module boundaries, typed interfaces and consistent patterns let an agent make a local change without understanding everything. Tangled code with implicit rules produces plausible changes that break something three modules away. Write the conventions down where the agent will read them:

# AGENTS.md

## Commands
- Install: `pnpm install`
- Test one package: `pnpm --filter <name> test`
- Type check and lint before finishing: `pnpm check`

## Rules
- Do not edit files under `migrations/` that are already merged.
- New endpoints need a contract test in `tests/contract/`.
- Never change a test only to make it pass. Explain the failure instead.
- Ask before adding a dependency.

A repository that is easy for an agent to work in is easy for a new colleague too. The work is the same.

Guardrails: permissions, sandboxes and secrets

An agent with a shell can do anything the user account can do. Limit that before the first incident.

Sandbox the execution. Claude Code’s documentation describes an optional sandbox enforced by the operating system (Seatbelt on macOS, bubblewrap on Linux) that confines writes to the working directory and routes network access through an allow-list. It also says plainly that this should not be the only control. Cloud agents run in disposable environments, which is safer by default, though they still need network limits.

Use the platform’s protections. GitHub’s documentation for its cloud agent lists several worth copying anywhere: the agent pushes to a single branch, it cannot approve or merge its own pull request, the person who asked for the work cannot approve it either, CI workflows do not run on its code until a person has reviewed it, and internet access is restricted by a firewall.

Keep secrets away. No production credentials on machines or in environments where agents run. Give the agent short-lived tokens with the narrowest scope that lets it do the task, and scan its commits for secrets like anyone else’s.

Assume prompt injection. An agent reads issue text, web pages, dependency documentation and error messages, and any of them can contain instructions aimed at it. The defences are structural: limited network egress, human approval for destructive commands, protected branches and a reviewer who reads the diff. We cover the pattern in guardrails for AI agents, and the same engineering applies to agents in other domains.

Where junior and senior roles shift

Senior engineers spend less time writing code and more time deciding what should be written: splitting work into pieces an agent can finish, specifying acceptance criteria, reviewing and keeping the architecture coherent. Those were always the scarce skills. Agents make the shortage more obvious.

The risk is with juniors. Much of what used to teach a new developer, such as the small bug fix or the tedious migration, is exactly what agents do well. A junior who only prompts and merges learns little and cannot review what they did not understand. We would deal with that on purpose. Have juniors explain an agent’s diff line by line in review. Give them ownership of a service, including being on call for it. Keep some work agent-free so the debugging muscle develops.

We would not stop hiring juniors. A team with no one learning the system has no seniors in five years.

Introducing agents without a quality drop

Take a twelve-person product team with a five-year-old monolith and patchy tests. Handing everyone an agent on Monday would produce a flood of pull requests against code that cannot verify them. A slower sequence works better.

  1. Weeks one and two: read-only and tests. Use agents to explain code, trace bugs and write tests for the areas you plan to change. Nothing risky merges, and the safety net improves.
  2. Write the conventions file. Commands, rules and the things that must never be touched. Update it every time an agent makes a mistake a rule would have prevented.
  3. Weeks three to six: small labelled tickets. Mark tickets as agent-ready only when they have clear acceptance criteria and live in tested code. One developer steers each task and stays responsible for it.
  4. Set the review rule. Every agent-written change gets a human reviewer who did not prompt it, and CI must pass. No exceptions for small changes.
  5. Then background agents for chores. Dependency updates, lint debt and flaky test fixes are good first candidates, in a sandboxed environment with scoped credentials.
  6. Review monthly. Look at what was reverted, what review missed and what the conventions file still lacks.

Throughout, one rule holds: the person who merges a change owns it. Saying that the agent wrote it explains nothing in an incident review.

How to measure the effect honestly

This is harder than vendors suggest, and the best evidence shows why. METR ran a randomised trial in early 2025 in which experienced open-source developers took 19 per cent longer on tasks when AI tools were allowed, while believing they had been faster. METR now marks that result as out of date. In its February 2026 update it said developers are probably faster with current tools, and also that its newer data is unreliable, partly because many developers no longer wanted to do tasks without AI and held back the tasks where it helps most.

Two lessons follow. Self-reported speed is a poor guide. And even careful researchers struggle to put a number on the effect, so be wary of any single percentage.

Measure outcomes on your own team instead:

  • lead time from first commit to production
  • time pull requests wait for review, and time spent reviewing
  • change failure rate and time to restore service
  • rework: the share of merged changes reverted or patched within two weeks
  • defects found after release

Compare the same team before and after on similar kinds of work, over months and not days. Do not use lines of code, pull request counts or the share of code written by AI. All three rise whether or not anything useful happened.

DORA’s 2025 research describes AI as an amplifier of an organisation’s existing strengths and weaknesses. That matches what we see. Good tests, fast pipelines and clear ownership get more from agents. Their absence gets amplified as well. The pipeline side of that is platform engineering work, and it comes first.

Where to start

In the first week, pick one repository with decent tests, write its conventions file and have two developers use an agent on real tickets with every change reviewed by someone else. Record the baseline numbers above before you begin.

The first decision is not which tool. It is what an agent may touch without a person watching, and who reviews the result. Start narrow, widen as the reverted-change rate stays flat, and evaluate agents the way you would any automated worker, as described in how to evaluate AI agents. Setting up this kind of agent-assisted delivery is part of our product engineering practice.

Questions

Asked often.

What is an AI coding agent?

An AI coding agent is a tool that takes a task described in plain language, reads the relevant parts of a codebase, edits files, runs commands such as tests and linters, and repeats until the task is done. Many can open a pull request for a person to review. That is different from autocomplete, which only suggests the next few lines.

Will AI coding agents replace developers?

Not on the evidence so far. Agents shift where the effort goes: less typing, more specifying, reviewing and deciding. Someone still has to understand the system, break work into safe pieces, judge whether a change is correct and answer for it in production. Teams need fewer hours of routine coding and more engineering judgement.

Is it safe to let an AI agent run commands in our repository?

It can be, with limits. Run the agent in a sandbox or a disposable cloud environment, restrict network access, keep production credentials out of reach, and require human review before anything merges. Treat issue text, web pages and third-party code the agent reads as untrusted input, because hidden instructions in them can steer it.

How do we measure whether coding agents make our team faster?

Measure outcomes, not activity. Track lead time from first commit to production, review time, change failure rate and how often merged work is reverted or patched soon afterwards, and compare the same team before and after on similar work. Developers' own impressions of speed and counts of generated lines are unreliable guides.

Not on the list? Ask us directly

Keep reading

More notes from the build.

All blogs

Let’s build your stack.

Send a brief and we’ll reply with next steps.