Risk-Tiered Review: The 2×2 That Decides When Humans Should Look

Module 20: Human-in-the-Loop Placement · Expansion Guide

Back to Module 20

The Problem

Pick the version that's yours.

TEAM: You shipped agents that ship code. Your team is split into two camps. Camp A wants every agent commit reviewed by a human, full stop, every time. Camp B wants auto-merge or the agent isn't worth its tokens. The fight is now slowing the team down more than the agent ever did.

SOLO: You shipped one agent that drafts customer responses for you. Now you can't decide which drafts deserve a 30-second review before sending and which can go straight out. You're either reviewing 80 a day (slow) or sending 30 unreviewed and missing the bad one (sloppy).

BOT: Your customer-facing AI generates invoice descriptions. Should every output go through a critic agent, or can low-confidence ones get a “human will verify” footer and high-confidence ones ship as-is? You've shipped both extremes; neither feels right.

Three modes, one decision: reversibility × blast radius decides the review tier. The actor decides who the reviewer is.

The Core Insight

Reversibility × blast radius is the only matrix that matters. Effort, sophistication, “how hard is this to review” are noise.

Most teams pick review tiers by feel — this looks risky, gate it; this looks small, ship it. By feel doesn't scale, drifts as people churn, can't be enforced, and gives no audit trail when something goes wrong. By feel is how you got here.

Two axes give you a defensible, enforceable rule. Reversibility: if this is wrong, can you undo it cheaply, or are you stuck with the result? Blast radius: if this is wrong, how many users, systems, or downstream things does it touch? Plot every kind of agent-driven action on those two axes and three review tiers fall out cleanly.

// across actors

TEAM: Reviewer placement — who looks at which tier, which work goes straight to ship.

SOLO: You-reviewing-future-you. Define the work that's auto-ship vs. work that needs your eyes before merge.

BOT: Critic agent in the loop on high-stakes outputs; pass-through on low-stakes.