Mastering AI agents
AI Ignoring Your CLAUDE.md or AGENTS.md? Turn the Rules Into Tests and Hooks (2026)
A rules file is advice the model reads. Which rules belong in a test that fails, which belong in a report a person reviews, and which belong in Claude Code and Codex hooks and permission rules, with real examples from my site.
Kelvin Wang · · 11 min read
Key points
- Write every rule down clearly first; only a clear rule tells you what should enforce it.
- Zero-exception rules a program can check become tests; a failing test means the work isn't done.
- Rules that need judgement become audit scripts that only report; a person decides.
- The AI's own actions, like deleting data or reading secrets, are stopped by hooks, permission rules and the sandbox, not by the model's memory.
Move the rules that must always hold out of the rules file and into checks a program runs. CLAUDE.md and AGENTS.md are read by the model, which usually follows them but isn’t guaranteed to. So I sort rules into three kinds: zero-exception rules a program can check become tests, and the work isn’t done until they pass; rules that need judgement become audit scripts that list what they find for me to review; and the AI’s own actions, such as deleting rows from a database, go to the tool’s hooks and permission rules, which stop them before they happen.
Why does AI ignore the rules in CLAUDE.md or AGENTS.md?
Because a rules file is context the model weighs, not a constraint it can’t cross. Claude Code’s best-practices page splits instructions into two kinds. What you put in a prompt, CLAUDE.md or a skill is read by the model, which then decides what to do; it usually follows, with no guarantee. What you configure as a hook, a deny rule or a managed setting is applied by Claude Code, the harness, every time, whatever the model decided 1.
The same page says that if Claude keeps ignoring a line even after you add IMPORTANT, there are usually two reasons: the file is long enough that the line gets lost, or the line is a rule with no exceptions and belongs in a hook 1. Codex works the same way: AGENTS.md is instructions assembled into the conversation, and the things that actually block are hooks, rules and the sandbox, covered below.
Before a rule can move anywhere, though, it has to be written clearly.
Turning rules into tests is really just spelling out, in words, the constraints I want.
“Keep the design consistent” can’t be tested. “State colours use only the four semantic roles; never write green-400 directly” can. Many rules only show who should enforce them once they’re written at that second level.
Which rules become tests, and which only get a report?
I sort with two questions. Does the rule have exceptions? When it’s broken, can a program tell?
| Kind of rule | Example | Enforced by | When broken |
|---|---|---|---|
| Has exceptions, depends on context | Don't copy the patterns in src/legacy/ | Rules file (AGENTS.md, CLAUDE.md) | The model judges |
| Checkable, but needs judgement | The first paragraph must not repeat the lead; question H2s end with "?" | Audit script tools/audit-writing.mjs | Listed in a report; I decide |
| Zero exceptions, checkable | Each exchange page has exactly one BreadcrumbList | Tests, npx vitest run | Test fails; not done |
| The AI's own actions | Dropping a table, truncating data, reading .env | Hooks, permission rules, sandbox | Blocked, or sent to me to confirm, before it runs |
This site’s repo has 81 test files. Most are ordinary feature tests; a set of them exists only to guard rules from DESIGN.md and the exchange-section spec. DESIGN.md is binding for the whole repo, and the first line of the project’s CLAUDE.md says to read it before changing anything visible. Rules and tests point at each other: the spec says “a test blocks this” and names the file, and the comment at the top of each test names the spec section it enforces.
What does a rule look like as a test?
Breadcrumbs are the clearest case. DESIGN.md says every public page except the home page gets one breadcrumb trail; the component prints both the visible nav and the schema.org BreadcrumbList, so pages must not write their own, and each page has exactly one.
The matching test is tests/exchanges-breadcrumbs.test.ts. It doesn’t sample a few pages. It finds every route file under src/pages/zh-hant/exchanges/, expands each into real URLs for every exchange and every incident, renders them one by one and checks three things:
- Exactly one
BreadcrumbList. One too many or one too few fails. - The first crumb is the home page.
- The last crumb is the page’s own URL.
A new exchange page is checked without touching the test. If the new page’s URL adds a new dynamic segment, the test fails outright and says where to add test values. “A new page forgot its breadcrumb” no longer depends on the AI remembering a rule.
The other spec-guarding tests follow the same pattern:
- State colours.
tests/status-colors.test.tsscans every component undersrcand fails on raw hue classes liketext-red-500orbg-green-400(DESIGN.md§6). - Spacing.
tests/spacing-scale.test.tsblocks vertical spacing on public pages that isn’t in the spec’s table (§4.10). - Exchange copy.
tests/exchanges-copy.test.tsblocks banned words, and copy may not contain literal percentages: fee rates are always filled into placeholders from approved data. - Incident page length. While an incident is live its copy is rewritten every hour and only grows, so
tests/exchanges-incident-copy.test.tscaps the summary at 3 sentences and 220 characters.
Exceptions live in the test too, each with a reason. The spacing test has an ALLOW table, with a note next to every file that keeps an off-scale value. The copy test’s percentage allow-list is keyed by file and JSON path, so any new percentage still fails. Banned words have no allow-list at all. Opening an exception means leaving a visible line in the diff.
Why do the writing rules only report instead of failing the build?
Because many writing rules are judgement calls. node tools/audit-writing.mjs scans every post and lists question H2s without a question mark, bullets without a bold lead, long posts without key points and so on, then always exits successfully. It blocks nothing. On 2026-11-17 it flagged 14 of the 160-odd MDX files.
“The first paragraph must not repeat the lead” is the best example. The script measures word overlap between the first paragraph and the description (character pairs for Chinese) and flags anything above 60%. That catches most copy-paste, but it also misfires, for example when two sentences share the same proper nouns and mean different things. Whether a paragraph really repeats the lead takes reading it.
Make that a failing test and one of two things happens: someone rewrites a paragraph that was fine just to get past the test, or exceptions pile up until the test means nothing. Claude Code’s best-practices page makes the same point: almost every guideline has exceptions, and with the code in front of it the model is usually better placed to judge whether one applies than a rule written in advance 1. So these rules produce a report and a person judges.
My rule: the script finds, a person decides. Every item in the report is mine to fix or leave.
How do Claude Code hooks and permission rules stop the AI’s actions?
Tests check what was produced. Some rules are about what the AI is doing right now, and by the time tests run it’s too late. Claude Code has two layers:
- Permission rules.
allow,askanddenyin a settings file. They are always evaluated in the order deny, ask, allow; the first match decides, and how specific a rule is doesn’t change that order 2. PutRead(./.env)under deny and Claude’s file-reading tools can’t read it. - Hooks. Shell commands you write that run at fixed points, such as
PreToolUsebefore every tool call. If the hook exits with code 2 or returnspermissionDecision: "deny", the call is cancelled and the reason goes back to Claude 3. A hook’s deny holds even inbypassPermissionsmode 3.
Both layers have documented limits. Read deny rules cover Claude’s built-in file tools and the commands it recognises, like cat and sed, but not a Python or Node script that opens files itself; for OS-level blocking of every process, the docs point to the sandbox 2. A prompt or agent hook asks another model to decide, so it isn’t deterministic either; for real enforcement, use a command or http hook 1.
The one I run is a PreToolUse hook on SQL that the AI sends through the Supabase MCP server:
- Destructive statements such as
DROPandTRUNCATE: go to Claude Code’s permission prompt with the full SQL shown, and run only after I approve. Under Remote Control the prompt reaches my phone. - A
DELETEthat would remove too many rows: raises inside the database and rolls back, so nothing is deleted. - Ordinary reads and writes: allowed, without interrupting the work.
Written only in CLAUDE.md, this rule would just be “ask me before deleting data”, and the model could decide this time doesn’t need asking. As a hook, whether to ask isn’t the model’s call anymore.
Does Codex have the same thing?
Yes, under different names. Codex has three things that can block:
- Hooks. The events look much like Claude Code’s, and
PreToolUsecan likewise returnpermissionDecision: "deny"or exit with code 2 to block a call. It covers shell commands,apply_patch(matchable asEditorWrite) and MCP tools, but not hosted tools such asWebSearch. Hooks go inhooks.jsonor[hooks]tables inconfig.toml, and hooks you add must be reviewed and trusted before they run 4. - Rules.
prefix_rule()written in Starlark matches on the start of a command and decidesallow,promptorforbidden; when several match, the most restrictive wins.codex execpolicy checktests a rules file. The docs mark the feature as experimental 5. - Sandbox and approvals.
workspace-writewithon-requestis the default for version-controlled folders: Codex can edit and run commands in the workspace, network access is off by default, and.git,.codexand.agentsstay read-only 6. That’s how my Codex is set up.
Codex’s docs are blunt about hooks: treat them as a useful guardrail, not a complete enforcement boundary. Some specialised tool paths can skip them, and with cloud orchestration a hook that errors, times out or returns a malformed response may let the tool run 4. So limits that truly can’t be crossed rely on the sandbox and permissions in both tools; hooks are an extra layer.
| What it controls | Claude Code | Codex |
|---|---|---|
| Rules file | CLAUDE.md (model reads it) | AGENTS.md (model reads it) |
| Check before a tool call | PreToolUse hook | PreToolUse hook |
| Command allow/deny lists | Permission rules allow / ask / deny | Rules: allow / prompt / forbidden (experimental) |
| OS-level isolation | Sandbox | Sandbox mode plus approval policy |
| When a hook fails | onFailure: "block" can block the action too | May still run (cloud orchestration) |
Sources: Claude Code best practices [1], permissions [2], hooks [3]; Codex hooks [4], rules [5], approvals and security [6].
When should a rule be upgraded to a test?
My order:
- Write it down clearly. In
DESIGN.md, the project’s AGENTS.md or CLAUDE.md, at a level a program could check: which values, which words can’t appear, how many per page. - Zero exceptions and checkable: write a test. Next to the spec line, note “a test blocks this” and the test file; in the test’s comment, cite the spec section.
- Checkable but needs judgement: write an audit script. It reports and blocks nothing.
- It’s about the AI’s actions: hand it to a hook, permission rule or the sandbox.
- When I point out one problem, treat it as a whole class. Scan every place with the same pattern and fix them together; if a program can find that class, it goes into a test, so next time it doesn’t depend on me spotting it.
On 2026-11-09 I asked for a “universal design rule”: whether Claude or Codex changes this site, brand colours, motion and corner radii stay consistent. It is still a proposal and hasn’t been built. The proposed direction is named tokens for values like radii and animation timings, with tests that block hard-coded values, rather than more paragraphs in the spec. A rule that only exists as prose has to be gambled on again with every new agent and every new session.
One caveat: when this site deploys, Cloudflare only runs the build (astro build), not the tests. npx vitest run is something the AI runs itself before committing, so “run the tests before you commit” is itself one of the rules in the rules file.
FAQ
Do rules that have tests still need to be in the rules file?
Yes, but shorter. The rules file tells the AI the rule exists, why, and where the test is; the test stops it when it forgets. With a test and no explanation, the AI only learns about the rule after the test fails, which costs an extra round. For how to write the rules file itself, see how to write CLAUDE.md and AGENTS.md.
Won’t the AI just edit the test to make it pass?
Test files are in the diff like any other code, so a change shows. That’s also why every exception has to go in an allow-list with a reason: one more exception is one more line the reviewer sees.
When I review, I pay attention to changes in the tests, but day to day I mostly look at the results: all green, and it passes.
Where do I start if my project has no test framework?
With a script that can fail. grep for banned words or hard-coded colour values and exit non-zero when it finds any, then run it before commits or in CI. Move to a real test framework once the rules pile up. For writing the design rules down in the first place, see design specs for AI-built websites.
Sources
Kelvin Wang
Electrical engineer building tools with AI agents, sharing hands-on notes on engineering and investing.
More about me