# Pattern Consistency Audit — and the inference test that proves it

Your agent copies whatever it already sees in your code. Show it three different answers and it will
invent a fourth. That is not carelessness — it is the agent *reporting* that your code has no answer
to copy.

Everybody accepts this. Almost nobody does the second half, which is the only half that is falsifiable:
**make the agent prove it can infer the pattern, with no help, and check the output.**

This skill does both. Step 1–2 find the pattern and the outliers with real counts. Step 3 runs the
test. A repo that passes Step 3 is done. A repo that fails it has a pattern the *author* can describe
and the *code* cannot — which is the same as not having one.

## Step 0 — Find the concerns. Do not interview the user, and do not invent them.

A "concern" is a thing the repo does many times: how a route is defined, how an error is handled, how
data is fetched, how a component is declared, how a test is written. **Which concerns exist is a fact
about this repo, and you read it off the tree.** Never assume a concern that has no files.

```bash
git ls-files | sed 's|/[^/]*$||' | sort | uniq -c | sort -rn | head -25
git ls-files | sed -n 's/.*\.\([a-zA-Z0-9]\{1,5\}\)$/\1/p' | sort | uniq -c | sort -rn | head -12
```

The directories with the most files are the concerns. A directory with **one** file is not a concern —
a pattern needs a population. Take the top three directories that hold ≥ 4 files of the same type, and
name the concern each one represents in the reader's own words.

Then check what the agent already reads, because it changes what you are testing:

```bash
ls CLAUDE.md AGENTS.md .cursorrules .github/copilot-instructions.md 2>/dev/null
ls .claude/rules/ .claude/skills/ 2>/dev/null
```

**Write down whether a CLAUDE.md exists. Step 3 removes it.** If the reader's pattern only survives
because it is written down in prose, the code has not got a pattern — the *file* has, and the file will
drift from the code the moment someone stops updating it.

## Step 1 — Count the competing patterns. Real numbers, not impressions.

For each concern, enumerate the *competing ways* the repo does it, then count each one across the whole
population. Structural search beats text search here, because `getUser` and `get_user` and
`const getUser = async () =>` are three different questions.

Install the structural matcher (ast-grep v0.44.0, npm `@ast-grep/cli`, https://ast-grep.github.io/reference/cli.html — fetched 2026-07-14):

```bash
npx --yes @ast-grep/cli --version
```

Run one structural pattern per competing form. `$VAR` is a wildcard for any single AST node, `$$$` for
any list of them, so the pattern is written as ordinary code:

```bash
npx --yes @ast-grep/cli run --lang ts --pattern 'export async function $NAME($$$) { $$$ }' --json=compact <dir> | node -e "let s='';process.stdin.on('data',d=>s+=d).on('end',()=>{const m=JSON.parse(s||'[]');const f={};for(const x of m)f[x.file]=(f[x.file]||0)+1;console.log(Object.keys(f).length,'files ·',m.length,'matches');console.log(Object.keys(f).sort().join('\n'));})"
```

Repeat, changing only the pattern, for each rival form you can see in the tree (a default export, an
arrow const, a class method, a decorated handler — read three files and the rivals will be obvious).
Where the language has no ast-grep grammar, or the difference is not structural (naming, import style),
fall back to a counted text sweep — the lesson's own fast version:

```bash
rg -c --no-heading -g '!node_modules' 'fetchUser|getUser|retrieveUser|loadUser' | sort -t: -k2 -rn
rg -n --no-heading -g '!node_modules' -o '\b(get|fetch|retrieve|load|read)[A-Z]\w+' | sed 's/.*://' | sort | uniq -c | sort -rn | head
```

Now compute the number that decides everything:

> **dominance = files matching the top pattern ÷ files in the population**

The lesson sets the bar and it is not 1.0: *aim for four files in five looking alike, not uniformity.*

| dominance | What it means | What to do |
|---|---|---|
| **≥ 0.80** | The code has an answer. The agent will copy it. | Fix the outliers, then go to Step 3. |
| 0.50 – 0.79 | A majority, but a contested one. The agent will follow it *sometimes* — which is worse than never, because you stop checking. | Pick the winner, converge the rest, re-count. |
| **< 0.50** | **There is no pattern.** Three answers to one question, so the agent invented a fourth. This is not the agent's bug; it is the finding. | Pick the example you would be happy to own five copies of. Converge. Do not proceed to Step 3 — you would be testing nothing. |

**Whatever shape is most repeated in the code IS the standard, whether the reader chose it or not.** If
the dominant pattern is the one they hate, say that out loud — it is the single most useful sentence in
this report, and it explains every surprising thing the agent has done to them.

## Step 2 — The outliers

The minority files are the ones actively teaching the agent the wrong thing. List every one, with the
path and the line, and classify it against the lesson's five situations:

| Situation | What you are looking at |
|---|---|
| **The Invented Fourth Way** | A file matching none of the rival patterns. Usually the agent's own last output. |
| **The Clever One-Off** | A file that is slick, compact and unlike everything around it. The agent either won't reuse it or will over-apply it. |
| **Four Names for One Thing** | The naming sweep found 3+ verbs for one job. |
| **The Half-Finished Move** | Two patterns split cleanly by file *age* — `git log --diff-filter=A --format=%ad --date=short -1 -- <file>` per file will show it. A migration that stopped halfway doubled the patterns instead of replacing one. |
| **It Copied Your Worst File** | The dominant pattern IS the mess. The code already voted. |

```bash
# the half-finished-move test: birth date of every file in the concern, next to which pattern it uses
git ls-files '<dir>' | while read -r f; do printf '%s  %s\n' "$(git log --diff-filter=A --format=%ad --date=short -1 -- "$f")" "$f"; done | sort
```

For each outlier the recommendation is always the same shape, and it is never "hand-edit the agent's
output" — that just adds a fifth way:

> *"Rewrite these N files so they follow exactly the same structure as `<the exemplar>`. List every
> difference you removed. Change no behaviour."*

## Step 3 — THE INFERENCE TEST. This is the part nobody does.

Everything above is a description. This is a **measurement**: can an agent that has never seen this
project, and has been told nothing, produce a file that matches?

The test is only valid if it is *clean*. Three rules, and each one exists because breaking it makes the
test lie:

1. **No CLAUDE.md, no rules files, no skills.** If they are present you are testing the reader's
   documentation, not their code. The lesson's whole claim is that *two matching examples beat one
   perfect explanation* — you cannot check that with the explanation still in the room.
2. **Only the exemplars.** Two or three files, the ones carrying the dominant pattern. Not the repo.
3. **No guidance in the prompt.** "Follow whatever conventions you can see" is the *most* you may say.
   Any hint from you is a hint the agent will not have on Tuesday.

Build the sandbox:

```bash
node -e "const fs=require('fs'),p=require('path'),root=p.resolve('.claude/scratch'),t=p.resolve(root,'inference');if(!t.startsWith(root+p.sep))throw Error('unsafe scratch path');fs.rmSync(t,{recursive:true,force:true});fs.mkdirSync(t,{recursive:true})"
cp <exemplar-1> <exemplar-2> .claude/scratch/inference/     # the two files that carry the dominant pattern
ls -A .claude/scratch/inference/                            # MUST show only the exemplars — no CLAUDE.md
```

Run it three times, because the agent is non-deterministic and a 1-of-1 result is an anecdote
(`claude -p` = non-interactive print mode, https://code.claude.com/docs/en/cli-reference — fetched
2026-07-14):

```bash
cd .claude/scratch/inference
for i in 1 2 3; do
  claude -p --model sonnet --permission-mode acceptEdits --add-dir . \
    "Here are my existing files. Without me explaining anything about them, add a third that <does the new thing>. Follow whatever conventions you can see." \
    > run-$i.log 2>&1
  mv <expected-new-file> attempt-$i.<ext> 2>/dev/null || echo "run $i produced no file"
done
```

Then score each attempt with **the same detectors from Step 1** — not by eye. That is what makes this
falsifiable:

```bash
npx --yes @ast-grep/cli run --lang ts --pattern '<the dominant pattern>' --json=compact .claude/scratch/inference/attempt-1.ts
rg -n --no-heading '<the dominant naming rule>' .claude/scratch/inference/attempt-*.ts
```

**The three outcomes, and how to read them:**

| Outcome | Verdict |
|---|---|
| The attempt matches the dominant pattern, and the agent asked nothing | **PASS.** The pattern is visible in the code. This is the end state, and it is the only proof that exists. |
| The attempt is generic — different names, different structure, reaching for a library the repo does not use | **FAIL.** The pattern is in the reader's head, not in their code. Go back to Step 2 and converge the outliers. |
| The agent **asked which approach to follow** | **FAIL, and this is the most useful failure.** That question is the finding: it is the code admitting it has more than one answer. Grep the run log for it — `rg -i 'which|should i|do you want|prefer' run-*.log`. |

Report the **pass rate: N of 3.** 3/3 is a pattern. 1/3 is a coin flip that happened to land your way
once, and that is exactly what "it works sometimes" felt like before you measured it.

Then run the second, cheaper test — the read-it-back:

```bash
cd .claude/scratch/inference
claude -p --model sonnet "Describe the pattern these files follow, as if writing it down for someone new. Change nothing." > described.md
```

If the description names a rule the reader does not recognise, **that is the rule the agent has been
following all along** — and it explains every output that has surprised them. Put it in the report
verbatim.

Clean up. The sandbox is scratch, not a deliverable:

```bash
node -e "const fs=require('fs'),p=require('path'),root=p.resolve('.claude/scratch/inference'),t=p.resolve(root,'.claude');if(!t.startsWith(root+p.sep))throw Error('unsafe scratch path');fs.rmSync(t,{recursive:true,force:true})"
```

## Step 4 — Emit the report

Write `pattern-audit.md` in the repo root. **Every path in it is a real path from Step 0's census.**
No invented file names, ever — an example file that does not exist in this repo makes the whole report
unreadable, because the reader cannot check a single row of it.

```markdown
# Pattern audit — <repo> — <date>

Concerns audited: 3 (from the file census). CLAUDE.md present: yes — REMOVED for the inference test.

## Concern 1 — <how the repo does X> (<n> files)

**Dominant pattern:** <the shape, written as code>
**Dominance: 11/14 files = 0.79** — below the 0.80 bar. Contested.

| Pattern | Files | Share |
|---|---|---|
| <form A> | 11 | 0.79 |
| <form B> | 2 | 0.14 |
| <form C — the agent's last output> | 1 | 0.07 |

### Outliers

| File:line | Situation | Why it teaches the wrong thing | Fix |
|---|---|---|---|
| <real path>:<line> | The Invented Fourth Way | matches none of A/B/C | rewrite to match <exemplar> |
| <real path>:<line> | The Clever One-Off | 9 lines doing what 30 lines do elsewhere; the agent has never once copied it | rewrite it the boring way |
| <real path>:<line> | Four Names for One Thing | `get*` ×9, `fetch*` ×3, `load*` ×1 | pick `get`; rename 4 call sites |

## Inference test — the measurement

Sandbox: 2 exemplar files, no CLAUDE.md, no rules, no skills. Prompt: "…without me explaining anything…"

| Run | Produced a file | Matched the dominant pattern | Asked a question |
|---|---|---|---|
| 1 | yes | **yes** | no |
| 2 | yes | no — used <form B> and a library not in package.json | no |
| 3 | yes | **yes** | no |

**Pass rate: 2 / 3.**

**Verdict: the pattern is not yet inferable.** One run in three reached for a different shape. That is
the same one-in-three the reader has been hand-correcting all week. Converge the three outliers above
and re-run this test — it is the only thing that will tell you it worked.

### Read-it-back

The agent, given only the exemplars, described the pattern as:

> "<verbatim from described.md>"

<If this is not the rule the reader thinks they have, say so here, in one sentence, and let it land.>
```

## Hard rules

- **Never invent an example file.** Every path in this report is real and came out of `git ls-files`.
  A report the reader cannot check against their own tree is worse than no report.
- **Never hand-edit the agent's output to make it match.** That adds a fifth way. The fix is always in
  the *code being copied*, not in the copy.
- **The inference test runs with the CLAUDE.md removed.** Otherwise you are grading the reader's prose.
- **A dominance below 0.50 means do not run Step 3.** The test would fail for a reason you already know,
  and you would have spent three model calls to learn it.
- **Report the pass rate, not a pass.** 3/3 and 1/3 are different findings and only one of them is done.
- **Mark deliberate exceptions in the code**, with a comment above them, so the agent does not copy them
  as a new example. An unmarked exception is just an outlier with a good story.

## What this skill will not do

- **It will not rewrite your files.** It counts, it names the outliers, and it hands you the convergence
  prompt. Rewriting eleven files to match a twelfth is a change with blast radius, and a human signs off.
- **It will not write your CLAUDE.md.** Writing the rule down is what you do *after* the code cannot say
  it by itself — and a rule you repeat by hand drifts. That is `vc-claude-md-harvester`.
- **It will not untangle legacy code whose rules live in people's heads.** That is a different job with a
  different failure mode (the knowledge was never in the repo at all).
- **It will not enforce uniformity.** Four files in five. The fifth one is allowed to be different, as
  long as it says so.
