Mastering AI agents
How Do You Verify an AI Agent's Work? 5 Checks After Claude Code or Codex Says Done (2026)
Claude Code or Codex says it's finished. How do you know it is? My definition of done for AI agents, the post-push live check, screenshot sizes and SEO follow-up, plus a template to paste into AGENTS.md.
Kelvin Wang · · 9 min read
Key points
- Done needs evidence: which command ran and what it printed, not "should be fine now".
- A successful push only means the code reached GitHub; it's live once the build passes and curl finds the change on the live page.
- Check screens at phone 390×844 and desktop, light and dark; log SEO changes after they ship and look back at 7, 14 and 28 days.
- Publishing an article, pushing to main and posting on social each need my explicit OK.
I split “done” into five pieces of evidence: local tests and the build pass, the deploy actually succeeded, the live page contains the change, phone and desktop screenshots look right, and SEO changes are logged and measured weeks later. Only when all five line up does an AI’s “done” count, and anything that gets published still waits for me to read it and say yes. The checks are the same whether the agent is Claude Code or Codex.
Why doesn’t “done” from an AI count?
An AI agent stops when the work looks finished. The Claude Code docs put it plainly: if there’s nothing it can run that says whether the work actually is done, “looks done” is all it has, and you end up being the test suite, with every mistake waiting until you happen to notice it1. The Codex docs say the same thing from the other side: Codex can run the change-test-confirm loop itself, but only if it knows what “good” looks like2.
So verification is less about checking hard afterwards and more about giving the AI something that comes back pass or fail before it starts:
- A test file or a build command. Exit code 0 passes, anything else fails. No room for interpretation.
- A command that compares output. For example,
curlthe live page and look for the sentence you changed. - Screenshots. Fixed sizes and themes, so it can see layout breaks for itself.
The Codex docs list not telling the agent how to run build and test commands as a common first-time mistake2; the Claude Code docs say that if a repo already has a check, the single most useful line in the rules file is the command that runs it1. The test command is the first thing I write in every project’s rules file.
Go deeper What Is an AI Harness? 7 Parts That Keep AI Agents Reliable (2026) Verification is the seventh part; the other six are rules, tests, skills, MCP, memory and subagents.Which checks run locally first?
This site’s rules file and my feedback to the AI name two commands it runs after any change:
npx vitest run: every test, including the site build tests, the article layout rules, the spacing scale, and rules that were turned into tests because they broke once.npm run build: a full build. Tests can pass while the build fails, for example when a component only errors at build time.
The report I want is simple: which command ran, how many passed and how many failed. “All tests pass” alone isn’t enough; I want the command and the numbers. When something fails, the fix is to read the error and fix it, not to skip the test.
When a rule is broken a second time, I turn it into a test. Then verification no longer depends on the AI remembering, or on me remembering: every npx vitest run checks it.
The push worked, so why wait for the deploy?
Because a successful push only means the code reached GitHub. Whether the site changed is a separate question. Since September 29, 2026 this site has been connected to Cloudflare Workers Builds: every push to main builds and deploys3. The build runs on Cloudflare’s machines, in a different environment from mine, so passing locally doesn’t mean it passes there.
On November 5, 2026, five of seven pushes in a row failed to build on Cloudflare with EMFILE: too many open files. Rollup, the bundler, opens up to 1,000 files at once by default, which exceeded the build machine’s open-file limit. Every push itself succeeded and GitHub had the new code, but the site stayed on the old version, so an update to a Binance incident page made during that stretch never went live.
The fix caps parallel file reads at 64 (commit 44ec11e), reproduced locally with ulimit -Hn 512, plus a test, tests/build-config.test.ts, so the setting can’t quietly disappear. Since that day my rules for AI agents include one more line: a push that hasn’t been confirmed live isn’t done.
After every push:
- Check the build result. Once Workers Builds is connected to GitHub, every build leaves a check run on GitHub4; read it with
gh api repos/<owner>/<repo>/commits/<sha>/check-runsand wait for success. - If the build failed, read the log and fix the cause. The log is in the Cloudflare dashboard’s build history3. Don’t push an empty commit to “try again”; that’s a gamble.
- Confirm the live page with curl (next section).
- Only then report it as live.
How do you confirm the live page actually changed?
The most direct way is to fetch the live page and look for the text you changed:
curl -s https://example.com/en/blog/<slug>/ | grep -c "the sentence you just added"
If it prints 0, it isn’t live, whatever the build status says. If the site sits behind a CDN cache and you see old content, rule out the cache before blaming the deploy.
Pages behind a login can’t be checked with curl. For this site’s /app dashboard, a script creates a signed-in session, fetches the JavaScript the dashboard actually loads, and checks it for a string that only the new version contains (tests/e2e/prod-bundle-check.mjs). Same idea: find something on the live site that only the new version has.
Which screen sizes should UI changes be checked at?
Tests catch logic errors. They don’t catch a button pushed off-screen on a phone. So for any visual change I have the AI take Playwright screenshots5, always the same four:
| Screenshot | Size | What to look for |
|---|---|---|
| Phone, light | 390×844 | Horizontal scrolling, tap targets big enough |
| Phone, dark | 390×844 | Text contrast in the dark theme, hard-coded white backgrounds |
| Desktop, light | 1280 wide | Two-column layout, table of contents, table widths |
| Desktop, dark | 1280 wide | Same, in the dark theme |
The phone rules come from this site's design spec, DESIGN.md §0.5 (phone rules) and §7 (checklist after UI changes); desktop at 1280 wide is a size I check on top of that.
Two things are measured with numbers instead of eyes:
- No horizontal scrolling. At 390px wide,
document.scrollWidthmust equal the viewport width. - Tap targets at least 44×44px. An icon can look small; the area you can tap can’t be.
Switch themes with the site’s own toggle, not only through the operating system setting.
After it ships, I open the live page myself. The screenshots an AI sends back don’t always show the part I want to look at.
How do you verify SEO changes?
An SEO change such as a new title or description can’t be judged the moment it ships; you have to wait for search data. My site is connected to an SEO Monitor that collects Google Search Console and GA4 numbers every day. Its rules live in the repo at seo/CLAUDE-SEO.md, so whichever AI tool works in this repo reads them.
- Log it only after it’s actually live.
record_changestores the URL, the type of change (title, description, content…), before and after, the reason, and the commit that shipped it. - Change one thing at a time. Change a page’s title and body together and you can’t tell which one moved the numbers.
- Look back at 7, 14 and 28 days.
get_change_historyfor that URL compares clicks, impressions and position before and after. - Check every language. A new Chinese title may mean the English version and hreflang need attention too.
For a small site, seven days of data is usually too little, and most changes are still inconclusive at that point, which is why the 14- and 28-day checks exist. Rankings move for many reasons at once, so a before/after difference is correlation, not proof of cause; I treat it as a hint, not a scorecard.
What needs a human sign-off?
Tests, deploy checks and screenshots can all run without me. These wait for my explicit OK no matter how thorough the checks were:
- Publishing a website article. I read every article myself and decide whether it goes out. The AI can write, run the tests and build, then stop and show me the result.
- Commits and pushes to
main. On this site a push deploys, so a push is publishing. Agreeing to “write an article” doesn’t include “publish it”. - Manual deploys. The AI doesn’t run deploy commands on its own; deploys go through the automatic build after a push.
- Threads and X posts. A separate question, with the exact post text in front of me. When I say “ready to publish”, I mean the website article, not social.
The split exists because a website article can be fixed after the fact, while a social post speaks to readers in my name and can’t be taken back once it’s out.
Code changes go through pull requests; I read the diff on my phone and decide whether to merge.
Go deeper How to Ship Code from Your Phone with Claude Code in 2026: Remote Control, Pull Requests and GitHub Mobile The AI branches, runs tests and opens a PR; I review the diff on my phone and merge.How do you write verification into AGENTS.md or CLAUDE.md?
Put the definition of done in the rules file, so the AI reads it at the start of every session and you don’t repeat it in chat. Codex reads AGENTS.md, Claude Code reads CLAUDE.md; I have both read the same file, as described in how to write CLAUDE.md and AGENTS.md. A template to adapt:
## Definition of done
- After a change, run `npx vitest run` and `npm run build`; report the commands and pass/fail counts.
- Visual changes: Playwright screenshots at phone 390×844 and desktop 1280 wide, light and dark.
- After a push, wait for the deploy to succeed, then curl the live page and find the change before reporting it live.
- If the deploy fails, read the log and fix the cause; don't push an empty commit to retry.
## Ask me first
- Commits and pushes to main (a push deploys).
- Any social post, with the exact text.
The rules file is the first step. A rule with no exceptions, such as “tests must pass before a commit”, can go further into a hook or CI, so the tool enforces it instead of the model choosing to follow it1.
FAQ
What if the project has no tests?
Start with the build: the exit code of npm run build is already a pass/fail check. Then pick the place that breaks most often and have the AI write one test that locks in the current behavior before changing anything. The Claude Code docs say that if a repo already has a check, the single most useful line in the rules file is the command that runs it1.
Can the AI check its own work?
Yes, but it’s better when the reviewer didn’t write the code. Claude Code’s /code-review reviews the diff in a fresh subagent that never saw the conversation, so it isn’t swayed by the reasoning that produced the code1. Codex’s /review can review against a base branch, uncommitted changes or a specific commit2. I add one more layer: anything with a clear answer, like tests and screenshots, goes to the tools; whether something gets published stays with me.
Isn’t four screenshots every time too many?
Text-only or backend changes don’t need screenshots; tests and the live check are enough. Anything that touches layout, components or CSS gets all four: phone and desktop use different layouts and the two themes use different colors, so one passing says little about the others.
Sources
Kelvin Wang
Electrical engineer building tools with AI agents, sharing hands-on notes on engineering and investing.
More about me