Home / University / Module 29 / Article

Audit, Compliance & Continuous Evals

Vibe Coder · 4.10 · Agent Systems

Back to Module 29

The Problem

Three questions you will be asked, in this order:

  1. From an auditor (during SOC 2 / ISO 27001 / HIPAA / DORA review): "Show me which parts of your system AI authored or operated, and the controls around it."
  2. From a customer (after a regression): "Was this regression introduced by AI? How do you catch them before they reach us?"
  3. From your own engineering team (during an incident): "Did the model drift? Was it a deploy? Both?"

Most teams can answer none of these. The reason is that the audit, compliance, and evals layers all evolved before LLMs were in production code. The frameworks predate the threat model.

The Core Insight

Three artifacts let you answer all three questions:

  1. An AI-touched manifest — a versioned record of which parts of the system are AI-authored, AI-operated, or both, and under what controls.
  2. Prod-traffic evals as a release gate — a small, real-shape sample is run through the new and old paths before promotion; quality regression blocks the release.
  3. A compliance posture — a one-page mapping from your existing controls (already in your SOC 2 / ISO scope) to the AI-specific question the auditor will actually ask.