A second agent is not a second opinion

The easiest way to make an AI coding workflow look rigorous is to add another agent.

One writes the patch. One reviews it. A third resolves any disagreement.

That creates more outputs. It does not necessarily create a new way to prove the patch wrong.

A recent GitHub feature makes the boundary concrete. On 17 July, GitHub announced that Copilot code review now reads custom instructions from a pull request’s head branch, including AGENTS.md, agent skills, and model-specific instruction files. This is useful: a team can test a review policy before merging it.

It also makes the trust boundary visible. The candidate branch can contain both the artifact under review and some of the context that shapes the reviewer.

That is not a claim that the feature is unsafe. It is a reminder that a reviewer is not independent merely because it runs in a different process or produces a separate comment.

Reviewer independence is a property of the evidence path, not the number of agents.

By evidence path, I mean the observations, checks, and decision rules that can cause rejection—and who controls them.

The practical question is therefore not “how many agents looked at this?” It is:

What could make the reviewer disagree with the builder, and did that reason come from somewhere the builder did not control?

Two agents, one assumption

Consider a retryable event consumer. One logical event should change a durable counter at most once.

Here is a deliberately small, constructed implementation:

def handle_non_atomic(store, event):
    if event.id in store.processed:
        return

    store.apply_effect(event.amount)
    store.processed.add(event.id)

Here store models durable state outside the worker. For this example, the failpoint belongs to the verifier’s harness, not the candidate handler: its store adapter can stop the worker immediately after apply_effect commits.

The code checks the processed set before applying the event. A candidate test can even deliver the same event twice and pass:

handle_non_atomic(store, event)
handle_non_atomic(store, event)
assert store.value == 5

A builder can claim that duplicates are handled. A reviewer anchored on the same successful trace can see the guard and the green test and approve.

Now change the trace, not the prose. The counter update succeeds, the worker crashes before persisting its deduplication checkpoint, and a fresh worker receives the same logical event:

import unittest


class FailureSetContractTests(unittest.TestCase):
    def test_named_wrong_fixture_is_rejected(self):
        trace = run_retry_trace(handle_non_atomic)

        self.assertEqual(trace.value_after_crash, 5)
        self.assertEqual(trace.effect_commits_after_crash, 1)
        self.assertNotIn("evt-7", trace.processed_after_crash)

        with self.assertRaisesRegex(ContractViolation, "AT_MOST_ONCE"):
            verify_at_most_once(trace)  # expected 5, got 10

The second agent did review the code. It just inherited the first agent’s evidence boundary. Their agreement was two executions of one interpretation, not two independent opinions.

I ran the complete example. It first confirms that the weak duplicate-after-success test stays green and that an atomic reference candidate passes. The same verifier then rejects the named wrong fixture with AT_MOST_ONCE: expected 5, got 10.

Its oracle checks both the final counter and the number of committed effects, closing the loophole where compensating writes happen to restore the expected final value. The executable version is small enough to inspect end to end.

The repair depends on the system boundary. If the effect and checkpoint share a store, an atomic transaction with a uniqueness constraint can close this window. If they do not, the effect sink may need to atomically enforce a stable idempotency key for the relevant retry horizon. Reversing the writes only trades the duplicate window for a loss window. That needs a separate liveness item in the failure set.

The design can vary. The review still had to imagine the crash window before any fix could be tested.

Review helps, but that is a different claim

Agentic review is useful. A July 2026 preprint, SWE-Review, evaluates 1,384 candidate pull requests derived from the 500 SWE-bench Verified issues. It reports that generate-review-revise loops raised resolve rates over one-shot generation. In a separate comparison, its repository-exploring reviewer outperformed single-turn, fixed-context review baselines in decision accuracy and resolve rate after revision.

That is useful evidence for review quality, not independent assurance. Its labels ultimately inherit SWE-bench Verified’s executable tests. OpenAI’s audit was deliberately failure-enriched: within 138 tasks that o3 failed to solve consistently across 64 runs, it found material specification or test problems in 59.4 per cent of that selected subset, alongside separate contamination evidence. That is not a benchmark-wide defect rate.

This does not erase SWE-Review’s comparative result. It narrows the claim: a review loop can improve a patch while its benchmark remains an imperfect oracle.

There is a useful current example of separating review paths. In its July audit of SWE-Bench Pro, OpenAI first used an automated filter to flag 286 of the 731 public tasks for deeper review. Within that subset, an agent-assisted path, with a researcher making the final judgment, classified 200 tasks as broken, while the human campaign identified 249.

The human reviewers first formed their own judgment from the visible problem statement, tests, and reference patch. Their category judgments later overlapped with the agent pipeline in 74 per cent of cases.

This does not prove that the human path was correct in every case. It shows why order and evidence ownership matter. Had reviewers begun with the pipeline’s conclusion, the exercise would have measured agreement with an anchor as well as the task itself.

Anthropic’s current agent-evaluation guidance similarly recommends composing code, model, and human graders because their strengths and failure modes differ. Composition is useful; a vote does not make their errors independent.

That is the central separation:

• Review quality asks whether another pass improves the artifact.
• Reviewer independence asks whether the pass brings a distinct way to discover that the artifact is wrong.
• Correctness asks whether the implemented behaviour satisfies the real requirement.

These properties can reinforce one another. They are not interchangeable.

Agreement has a family tree

Agreement is still evidence. Its strength depends on why the reviewers agree.

If two agents inherit the same task wording, examples, repository instructions, retrieved sources, model family, and test suite, their errors can be correlated. Changing the persona from “builder” to “senior reviewer” alters the prompt. It does not erase that ancestry.

This is not a new problem created by language models. In a 1985 technical report, Knight and Leveson’s multi-version programming experiment tested the assumption that independently developed implementations would fail independently. The observed failures were more dependent than the fault-tolerance argument assumed. That study involved human programmers and a specific task, not coding agents. The analogy is narrow: separate implementations do not automatically imply separate failure modes.

Current model research offers adjacent evidence, with important boundaries:

• A July 2026 single-author preprint, When LLMs Agree, Are They Right?, found agreement to be a positive but weak and regime-dependent predictor on GPQA and AIME. The study did not examine open-ended generation or code review.
• A NeurIPS 2024 paper found self-preference in LLM evaluators on summarisation tasks. It is evidence against treating same-model judgment as neutral, not a universal estimate of coding-review bias.

The safe conclusion is modest. Reviewer count is a poor proxy for evidence diversity.

Independence is a path property

Independent verification has a longer engineering history than agent review. NASA’s IV&V overview separates technical independence from managerial control over what is analysed, how it is tested, and which findings are pursued. Most pull requests do not need a NASA programme. The smaller lesson is that independence comes from control over the review method, not a new label on the same method.

For an agent-authored change, I would inspect four paths:

Different models, providers, prompts, and tools can improve diversity. None earns an “independent” badge alone. The question is whether their routes to error are meaningfully different for the risk being reviewed.

A failure-set contract

Before implementation, add a small failure-set contract: for each consequential invariant, name a plausible wrong program, the smallest trace that exposes it, and the rejection the verifier must produce.

This is not a new testing discipline. The proposal is organisational: combine mutation testing, property testing, fault injection, and independent verification in a workflow that derives and ratifies failure modes before the patch exists.

Mutation testing usually begins with existing code and injects small artificial faults to test the test suite. Google’s industrial mutation-testing work shows both its value and its scaling constraint: Google made it actionable by mutating changed code, filtering aggressively, and limiting the mutants shown during review.

A failure-set contract begins before implementation. It can describe a sequence-level domain error such as “effect commits, checkpoint does not” even when no small syntactic mutation captures it. The original QuickCheck paper showed how executable properties can be tested over random inputs and custom generators. That is a natural execution mechanism when the invariant can generate many traces. Fault injection, model checking, example tests, or a deterministic state check may fit better elsewhere.

The wrong program becomes part of the specification of the verifier. That gives “independent review” an observable meaning: the review path must be able to reach a rejection using evidence the builder cannot silently change.

Turn the contract into a gate

The contract needs an executable form. For the retry example:

id: duplicate-after-side-effect
invariant: one logical event changes the counter at most once
trigger:
  - deliver evt-7
  - persist +5
  - crash before persisting the dedupe checkpoint
  - redeliver evt-7
plausible_wrong_program: allows the effect to become durable without
  atomically recording the event id
expected_rejection: 'AT_MOST_ONCE: expected 5, got 10'

The gate has two halves. The candidate must satisfy the adversarial trace. The verifier must also reject a known-wrong fixture or mutant through the predicted channel. Otherwise a malformed check can produce a reassuring green result while observing nothing useful.

I would run the contract like this:

1. A human owner publishes the requirement, boundaries, consequential invariants, and acceptance policy.
2. Before seeing the patch or its rationale, a cold reviewer derives a small set of plausible wrong programs and the smallest traces that expose them.
3. A domain owner ratifies each item as implied by the public contract, plausible, non-equivalent, and consequential. The ratified set is then versioned and frozen.
4. The builder sees the invariant, valid behaviour, and ratified failure classes. A protected verifier owns its implementation, oracle, fault injector, acceptance policy, and any retained concrete traces. The candidate branch cannot change those assets without separate review.
5. Each check is calibrated first. A known-conforming reference must pass. Its named wrong fixture must reach the required failpoint and intermediate state, then be rejected through the predicted assertion rather than an unrelated crash.
6. Only then does the candidate run through ordinary tests and the failure set. A person owns exceptions, changes to the frozen set, and the final merge decision.

Agents can propose the failure set. They should not be the only authority ratifying it. Otherwise the same interpretation can write the program, choose the mistakes worth testing, and grade its own omissions.

Measure unique evidence, not reviewer count

The review system needs its own evaluation.

Seed or retain known faults, then ask which path catches each one. Mutation testing can supply local syntactic faults. Incident traces, property counterexamples, and hand-designed failure sets can supply semantic and sequence-level faults.

I would track four measures with explicit units:

• Unique defect yield: known faults caught by exactly one path, divided by known faults exercised.
• Shared miss rate: known faults missed by every path, divided by known faults exercised.
• False-positive cost: review time spent resolving findings that are not contract violations.
• Oracle coverage: ratified consequential invariants with a non-model check, divided by all ratified consequential invariants.

“Three agents approved” is an activity metric. “The retry property killed the duplicate-after-effect mutant” is evidence.

The process should scale with consequence. A copy change may need ordinary review. A state transition, permission boundary, migration, concurrency path, or irreversible effect deserves a larger failure set and a more protected verifier. Independence is not free, so spend it where a correlated miss can matter.

The easy fixes do not settle it

Use more agents. This adds search breadth and can improve recall. If their evidence ancestry is unchanged, it can also add correlated votes.

Use another provider. Model diversity may help. Providers may differ substantially, but they can still inherit the same specification, examples, repository, and tool outputs.

Give the reviewer an adversarial persona. A better question often produces a better review. It remains prompt diversity unless the reviewer also owns a distinct source of evidence.

Put a human at the end. Human ownership is essential for intent and consequence. A person can still be anchored by the builder’s explanation or reduced to approving a green dashboard. The review order matters for humans too.

Trust deterministic tests. Tests are powerful precisely because they can disagree with the prose. They only verify the properties their authors encoded. OpenAI’s two SWE-bench audits warn that a deterministic grader can be perfectly repeatable while its tests still misrepresent the intended requirement.

Hide all the tests. Hidden requirements make evaluation unfair and debugging wasteful. Keep the requirements, invariants, valid behaviour, and failure classes public. Retain concrete traces only where they help detect overfitting to examples.

Where the argument stops

Independence is not competence. A cold reviewer can independently misunderstand the requirement. A property test can formalise the wrong property. A human can ratify a bad failure set. A clean evidence topology does not prove correctness.

The failure set is deliberately bounded. Passing the gate means the verifier rejected the named wrong programs and the candidate passed the named traces. It says nothing about mistakes nobody imagined.

Nor should it become ceremony. Equivalent mutants need to be retired. Low-consequence changes should not inherit a safety-critical process. The useful version is small, risk-scaled, and revised when real incidents reveal a better wrong program.

The shift is simple:

Do not count reviewers. Trace what each reviewer can see, what it can change, and what could make it disagree with the builder.

A second agent becomes a second opinion only when its conclusion actually rests on an evidence path the first agent cannot silently change.