AI can now produce software at a speed that would have sounded ridiculous a few years ago. An agent can inspect a repository, plan a change, edit several files, write tests, run tools and prepare a pull request before a human has finished reading the ticket.
That is real progress. It is also the easy part to measure.
Lines changed, issues closed and pull requests opened all look like productivity. They show activity and, sometimes, useful output. They do not tell us whether the software is safe to release, whether the change solves the business problem or whether the team understands the risk it has accepted.
This is the uncomfortable limit of the "software factory" idea. AI can manufacture output. It cannot decide which defects a business can afford.
That decision needs context about customers, operations, money, regulation, reputation and timing. It needs somebody who can distinguish an ugly but harmless edge case from a quiet error that corrupts invoices for months. It needs product ownership and engineering judgment.
Faster generation does not reduce the value of that judgment. It raises it.
What the software factory promise gets right
The factory metaphor is attractive for good reasons. Traditional software work contains plenty of delay that is not creative or valuable:
- searching a large codebase for the right extension point;
- repeating a familiar migration across several packages;
- writing routine tests and documentation;
- updating similar configuration in many places;
- reproducing a reported issue;
- preparing a first implementation for review.
Coding agents can compress much of this work. They can keep more context in motion than a person wants to hold at once, retry patiently and operate across tools without getting bored. A capable team can use that leverage to shorten feedback loops and spend less time on mechanical tasks.
The mistake is not using AI.
The mistake is turning a useful production tool into a management fantasy: feed requirements into one end, receive trustworthy software from the other, and remove the expensive humans in the middle.
The middle is where the business meaning lives.
An agent sees code, instructions, tests and tool output. It may also see documentation and selected business rules. It does not automatically know which rule is current, which exception came from a legal requirement, which customer promise is still active or which operational shortcut the company quietly depends on every Friday afternoon.
If that knowledge is not explicit, the factory cannot manufacture it.
Output is not the same as an accepted change
A generated change becomes useful only when someone can accept it.
Acceptance sounds simple until a team tries to define it. "The tests pass" is one signal, not a complete standard. Tests can be incomplete, outdated or aimed at the wrong behavior. A green build can still ship a confusing workflow, an inaccessible interface, a performance regression or a financial error nobody thought to encode.
An accepted change needs a clearer contract:
- The intended behavior is stated.
- Important failure cases are identified.
- Evidence shows that the implementation meets the contract.
- Known residual risk is visible.
- A person or role has authority to release it.
- Somebody owns the result after release.
This contract is useful with or without AI. Agents simply expose how often teams have been relying on informal understanding instead.
When code was expensive to produce, humans naturally limited the number of changes moving at once. Review capacity was still a problem, but generation itself acted as a brake. AI removes much of that brake. A team can now create changes faster than it can understand them.
The bottleneck moves.
The scarce resource is no longer keystrokes. It is reliable acceptance.
Every business already has a defect budget
No production system is perfect. Every business lives with defects, workarounds and deferred improvements. The important question is not whether defects exist. It is whether the organisation understands which ones it can tolerate.
Consider three hypothetical issues:
- A product image is slightly misaligned on one old mobile browser.
- A delivery estimate is wrong for one remote postcode.
- A rounding error changes tax totals on a subset of invoices.
All three are defects. They are not equally important.
The first may be annoying but acceptable until the next frontend release. The second may create support cost and disappointed customers, so it needs faster action. The third can create accounting, legal and trust problems even if very few users see it.
An AI model can help detect, reproduce and fix all three. It cannot set their business priority from code alone.
Priority depends on impact, reach, reversibility, detectability and timing. It also depends on promises the organisation has made. A bug that would be minor for a content site may be critical in checkout, identity, healthcare or financial reporting.
Teams need a simple language for this. Not a fifty-page risk framework that nobody opens, but a working defect budget:
- What must never be wrong?
- What may degrade temporarily?
- Which operations need a manual fallback?
- How quickly must each class of failure be detected?
- Who can accept a known issue?
- When does a small issue become a release blocker?
Without those answers, an agent will optimise for the evidence it can see. Often that means passing tests, satisfying a ticket and producing a plausible implementation.
Plausible is not the same as acceptable.
Acceptance criteria are business instructions
Many tickets describe what to build but say little about what must remain true.
"Add a discount field." "Move checkout to the new service." "Let users change their delivery date."
These instructions leave the expensive questions open.
Can discounts stack? What happens when a refund occurs? Which system owns the final price? Can a customer choose a date after stock allocation? What happens when the delivery partner rejects the change? Does the old flow still need to work for existing orders?
A human developer notices some of these gaps and asks questions. An agent may also notice them, especially if prompted to challenge the specification. Neither can guarantee that the right business owner will answer.
Good acceptance criteria are not ceremonial ticket decoration. They are business instructions written clearly enough to test.
They should cover the normal path, meaningful edge cases and the conditions that turn a change into a failure. They do not need to predict every possible event. They do need to name what the business cares about.
For a checkout change, that might include:
- the total shown before payment matches the amount charged;
- tax and discount rules remain consistent across web, invoice and refund;
- an interrupted payment does not create a duplicate order;
- stock is not committed twice;
- the customer receives a clear state when a provider is unavailable;
- support can identify and recover a stuck transaction.
These statements give both humans and agents something real to work against.
More tests can still test the wrong thing
AI makes test generation cheap. That is useful, but it can create false confidence.
An agent can generate hundreds of unit tests around the implementation it just wrote. Those tests may prove that the code behaves as designed. They may not prove that the design matches the business.
This is a familiar problem with a new scale.
If the requirement is wrong, generated tests can preserve the wrong behavior very efficiently. If the test data is too clean, the suite can miss production messiness. If every assertion mirrors the implementation, the tests may confirm structure rather than outcome.
The answer is not to distrust generated tests. It is to diversify the evidence.
A serious release may combine:
- deterministic unit and integration tests;
- contract tests between services;
- representative production-like data;
- accessibility and performance checks;
- adversarial review from a separate model or person;
- a manual walkthrough of the business-critical path;
- monitoring and rollback readiness after release.
The mix depends on the change. A copy update does not need the same evidence as a payment migration. Judgment decides the difference.
Review capacity becomes the real production limit
If agents produce ten times more changes, a team cannot respond by asking the same reviewers to read ten times more code.
That path ends in shallow review, rubber-stamp approvals and exhausted senior engineers. The organisation appears faster while its understanding of the system becomes thinner.
The review process has to change with generation speed.
First, not every change deserves the same route. Low-risk, reversible changes can use strong automated checks and lighter human review. High-impact changes need deeper evidence and explicit ownership.
Second, reviews should focus on risk and intent, not only syntax. A human does not need to admire every generated loop. The reviewer needs to know:
- why the change exists;
- what behavior changed;
- which assumptions were made;
- what could fail;
- how the result was tested;
- how to observe and reverse it.
Third, generated changes should arrive in reviewable pieces. A pull request that edits eighty files because the agent could do it is not automatically efficient. Smaller coherent changes make errors easier to isolate and intent easier to understand.
Finally, teams should measure review time and rejected output. If generation rises while acceptance slows, the factory is creating inventory, not value.
The person approving the release needs authority
Ownership is often treated as a project-management detail. In AI-assisted development it becomes a safety mechanism.
Someone must have authority to say:
"This evidence is enough." "This known issue is acceptable for this release." "This change needs another test." "We are not shipping this today."
If nobody owns that decision, the team drifts toward one of two bad defaults.
The first is paralysis. Every stakeholder is consulted because no one can accept the risk. Work waits in queues while the supposed factory keeps producing more.
The second is accidental release. A developer or agent merges because the build is green and no one objected. Silence becomes approval.
Neither is a technical problem. Both are failures of authority design.
The owner does not need to be one person for every change. Product, engineering, security, finance and operations may each own different decisions. The important part is that the boundary is known before release time.
Observability is part of the definition of done
Even excellent pre-release checks cannot predict everything in production.
That makes observability part of acceptance, not a task for later.
Before shipping a meaningful change, the team should know what success and failure will look like after release. Which metric moves? Which error appears? Which customer report indicates trouble? How quickly will someone notice?
An AI-generated change without production visibility creates a dangerous asymmetry. The organisation can introduce behavior faster than it can detect consequences.
For critical paths, the release packet should include:
- the expected production signal;
- the failure signal;
- the relevant logs or dashboard;
- the person watching;
- the rollback or containment action;
- the time window for evaluation.
This does not require an enterprise command centre for every deployment. It requires proportion.
A reversible visual change may need a quick check. A new pricing calculation needs stronger monitoring and reconciliation. A permission change deserves logs and an audit trail.
The same question applies every time: if this is wrong, how will we know?
Reversibility changes what the business can safely try
Fast generation becomes much more valuable when changes are easy to reverse.
Feature flags, staged rollouts, backups, versioned data migrations and tested rollback paths reduce the cost of learning. They let teams experiment without pretending uncertainty has disappeared.
But "we can roll back" should be tested, not assumed.
Some changes cannot be cleanly reversed. Data may have been transformed. Emails may have been sent. Customers may have acted on a price. External systems may have received instructions. A rollback can restore code while leaving the business consequence in place.
That is another judgment AI cannot make from a diff.
Teams need to distinguish technical reversibility from business reversibility.
A practical operating model for AI-assisted delivery
The following model is deliberately simple. It is meant to help teams use more automation without pretending every change is equal.
Start with the business outcome
Describe the customer or operational result before describing the implementation. "Customers can change delivery date without contacting support" is stronger than "add a date selector".
Classify the risk
Consider impact, reach, reversibility, detectability and sensitivity. A small content adjustment and a tax calculation should not share the same release path.
Define evidence before generation
Decide what must be true and how it will be demonstrated. This keeps tests from becoming a story written after the implementation.
Let agents do the mechanical work
Use agents for repository exploration, first implementations, routine tests, migrations, documentation and repeated checks. Give them clear boundaries and the tools needed for the task, not every credential available.
Separate creation from challenge
Have a different reviewer, model or process attack the assumptions. The system that created the change should not be the only system judging it.
Package the decision
The approver should receive a concise explanation of intent, changed behavior, evidence, known risk, monitoring and rollback. A large diff is not a decision document.
Observe the release
Check the production signals for an appropriate period. Capture what happened so the next change benefits from evidence rather than memory.
Metrics that matter more than generated code volume
If a company wants to know whether its software factory is working, it should resist vanity metrics.
Lines of code, agent runs and pull-request count can all rise while delivery quality falls.
Better measures include:
- lead time from approved need to accepted release;
- percentage of generated changes accepted without major rework;
- human review time per accepted change;
- escaped defects by impact, not only count;
- rollback and incident rate;
- time to detect and recover;
- change failure rate on business-critical paths;
- percentage of changes with a named owner and observable success signal.
These metrics are not perfect. No single number can represent engineering quality. Together they show whether automation is producing outcomes or merely output.
Engineering judgment is not a romantic defence of manual work
It is easy to turn this conversation into a false choice.
One side imagines fully autonomous factories. The other defends hand-written code as if typing were the source of quality.
Neither position is useful.
Engineering judgment does not mean humans must manually perform every task. It means humans remain accountable for framing the problem, defining evidence, understanding trade-offs and accepting risk.
The best use of AI is not to preserve old workflows unchanged. Teams should remove unnecessary ceremony, automate repetitive checks and let agents handle work that does not deserve scarce human attention.
Humans should spend more time where context and consequence matter.
That may mean less time writing boilerplate and more time defining a migration boundary. Less time searching logs and more time deciding what failure requires immediate action. Less time formatting tickets and more time resolving an ambiguous business rule before it becomes code.
This is not a smaller role. It is a sharper one.
Build a factory that can explain itself
The most trustworthy software factory will not be the one that produces the most code while everyone sleeps.
It will be the one that can answer, for every meaningful change:
Why was this built? What changed for the user or operation? What evidence supports the release? What risk remains? Who accepted it? How will we know if it fails? How do we contain or reverse it?
AI can help generate every part of that record. It can assemble evidence, summarize a diff, run checks and keep the decision packet current.
But the business still has to choose what good means.
That is the part no model can infer safely from output volume.
Software generation is becoming abundant. Acceptance, authority and judgment are not.
Companies that understand that distinction will move faster without losing control of what they ship.
Source 1: https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/wsff.md
Source 2: https://tldr.tech/tech/2026-07-24
