验收 · 怎么评测你委派的活Review · How to Evaluate What You Delegate.
派活容易，难的是确认它真做对了。这是编排时代唯一持续稀缺的技能 —— 会用 agent 的人过剩，能系统性验收 agent 的人不够。凭手感看两眼 diff 不算验收：你需要能复跑的检查（测试、类型、跑一遍看结果）、一套判断它是否跑偏的标准、以及把「这次错在哪」沉淀成下次能自动拦住的用例。这一章把你日常的「看一眼就合并」升级成一套真正的验收工作流，并接上《Agent 实战》里评测那一章 —— 这也是这本书和那本书真正合流的地方：Claude Code 是你派活的地方，评测是你确认没被坑的地方。Delegating is easy; confirming it actually got it right is hard. This is the one skill that stays scarce in the orchestration era — people who can use agents are plentiful, people who can systematically verify them are not. Eyeballing a diff isn't verification: you need repeatable checks (tests, types, actually running it), a standard for whether it drifted, and a way to turn 'here's what it got wrong this time' into a case that auto-catches it next time. This chapter upgrades your everyday 'glance and merge' into a real review workflow, and connects to the evaluation chapter in Agent in Practice — where the two books truly meet: Claude Code is where you delegate, evaluation is where you make sure you weren't burned.
10 分钟 · 初稿 2026.0710 Min · Drafted 2026.07
到第 5 章,你已经会把活派出去;到第 8 章,活甚至不在你眼前跑。于是只剩一个问题:交回来的东西,你怎么知道是对的?扫两眼 diff、看它说一句「测试通过了」、合并 —— 那不是验收,是祈祷。这一章讲编排时代唯一持续稀缺的技能:会派活的人已经太多,能系统性验收的人还太少。By chapter 5 you can send work out; by chapter 8 it doesn't even run where you can see it. One question remains: when the work comes back, how do you know it's right? Skimming the diff, reading its 'tests pass', merging — that isn't review, it's prayer. This chapter is about the one skill that stays scarce in the orchestration era: plenty of people can delegate to agents; few can systematically verify them.
— I
底线:一条能复跑的检查The Floor: One Rerunnable Check.
没有检查的时候,「看起来做完了」是它唯一的停机信号,而你是唯一的验证回路 —— 每个错误都躺在 diff 里,等你亲眼看见。所以 Anthropic 自己的最佳实践把这条排在第一:给它一个它能自己跑的检查 —— 测试、构建、一张能对比的截图。有了能返回过/不过的信号,回路才闭合:它干活、跑检查、读结果、修到过为止。1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17.Without a check, 'looks done' is its only stopping signal, and you are the only verification loop — every mistake lies in the diff, waiting for your eyes. Which is why Anthropic's own best practices put this first: give it a check it can run itself — tests, a build, a screenshot to compare. Once there's a pass/fail signal it can read, the loop closes: it works, runs the check, reads the result, and iterates until green.1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17.检查不必隆重。一条测试、构建的退出码、一个 linter、一个把输出和样例对拍的脚本、一张和设计稿并排的截图 —— 任何能压成过/不过的东西都算。再加一条纪律:要证据,不要断言。让它交测试输出、贴跑过的命令和返回,而不是一句「我搞定了」;看证据比你亲手重跑一遍快,也是你敢不盯着它的前提。1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17.The check needn't be grand. One test, a build's exit code, a linter, a script that diffs output against a fixture, a screenshot beside the design — anything that reduces to pass/fail counts. Add one discipline: evidence, not assertions. Have it hand over the test output, the command it ran and what came back — not a 'done!'. Reading evidence is faster than rerunning the verification yourself, and it's what lets you stop watching.1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17.
提示词Prompt派活时把验收一起派下去Send the acceptance criteria with the job
这件活的验收标准:<一条可检验的标准>。
做完后跑 <检查命令>,把完整输出贴给我。
没验证过的部分明说「没验证」,不要写「应该可以」。
检查跑不过就修到跑过再交,别把失败的输出藏起来。
最后列出你改过的每个文件和一句为什么。Acceptance criteria for this task: <one checkable criterion>.
When done, run <check command> and paste the full output.
Mark anything unverified as 'not verified' — never write 'should work'.
If the check fails, fix until it passes before handing off; don't bury failing output.
End with every file you changed and one line on why.
这件活的验收标准:登录超时后刷新 token 的用例全部通过。
做完后跑 npm test -- src/auth,把完整输出贴给我。
没验证过的部分明说「没验证」,不要写「应该可以」。
检查跑不过就修到跑过再交,别把失败的输出藏起来。
最后列出你改过的每个文件和一句为什么。Acceptance criteria for this task: all token-refresh-after-timeout cases pass.
When done, run npm test -- src/auth and paste the full output.
Mark anything unverified as 'not verified' — never write 'should work'.
If the check fails, fix until it passes before handing off; don't bury failing output.
End with every file you changed and one line on why.
— II
第二层:对着计划审,换个脑子审The Second Layer: Against the Plan, with a Fresh Mind.
检查证明「它能跑」,证明不了「是你要的」。跑得过测试的代码,也可能顺手重构了你没让它动的模块、绕开了你点名的方案。第二层验收要有基线 —— 最现成的基线,是计划模式里你批准过的那份计划(第 2 章):把 diff 对着计划逐条核,要求都实现了吗、点名的边界情况有测试吗、范围之外的东西动了没有。A check proves it runs; it doesn't prove it's what you asked for. Code that passes tests can still have refactored a module you never authorized, or side-stepped the approach you named. The second layer needs a baseline — and the readiest one is the plan you approved in plan mode (chapter 2): read the diff against it item by item. Every requirement implemented? The named edge cases tested? Anything outside scope touched?这层审最好换个脑子来。写代码的那个上下文装满了它自己的推理,官方文档说得直白:新上下文审得更好,因为它不会偏心自己刚写的代码。1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17.subagent 恰好提供这种隔离 —— 独立上下文、独立提示,只看 diff 和你给的标准,把发现交回主对话2注 2Note 2Claude Code Docs · Subagents —— 每个 subagent 在自己的上下文窗口里运行,带自己的系统提示与工具权限;独立干活,只把结果交回主对话。截至 2026-07-17。Claude Code Docs · Subagents — each subagent runs in its own context window with its own system prompt and tool access; it works independently and returns only results to the main conversation. As of 2026-07-17.;内置的 /code-review 就是这么做的:在一个全新的 subagent 里审当前 diff。1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17.This layer is best done by a different mind. The context that wrote the code is full of its own reasoning; the docs put it plainly: a fresh context reviews better because it 'won't be biased toward code it just wrote.'1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17. Subagents supply exactly that isolation — separate context, separate prompt, seeing only the diff and your criteria, handing findings back2注 2Note 2Claude Code Docs · Subagents —— 每个 subagent 在自己的上下文窗口里运行,带自己的系统提示与工具权限;独立干活,只把结果交回主对话。截至 2026-07-17。Claude Code Docs · Subagents — each subagent runs in its own context window with its own system prompt and tool access; it works independently and returns only results to the main conversation. As of 2026-07-17.; the bundled /code-review does precisely this: it reviews the current diff in a fresh subagent.1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17.
— III
固化:从「你记得看」到「机器替你拦」Codify: From 'You Remember' to 'the Machine Blocks'.
同一套手动验收做到第三遍,就该问:哪部分能固化?Claude Code 给了三个台阶,约束力一级比一级硬。第一阶是 CLAUDE.md:把「提交前要跑 lint」写进去,它每次开工都读 —— 但文档说得明白,这是上下文,不是强制配置,它大概率照做,不保证。3注 3Note 3Claude Code Docs · Memory —— CLAUDE.md 与 auto memory 都是「上下文,不是强制配置」;要不论模型怎么决定都拦住某个动作,用 PreToolUse hook。截至 2026-07-17。Claude Code Docs · Memory — CLAUDE.md and auto memory are 'context, not enforced configuration'; to block an action regardless of what the model decides, use a PreToolUse hook. As of 2026-07-17.第二阶是 hooks:用户定义的命令,在生命周期的固定时点自动执行 —— PostToolUse 在每次改完文件后跑格式化,PreToolUse 在它碰危险路径之前直接拦下,Stop 在它想收工时跑你的检查、不过不放行。4注 4Note 4Claude Code Docs · Hooks reference —— hooks 是「用户定义的 shell 命令、HTTP 端点或 LLM 提示,在 Claude Code 生命周期的特定时点自动执行」;事件含 PreToolUse、PostToolUse、Stop、SessionEnd 等,可拦下工具调用、改写输入输出、注入上下文。截至 2026-07-17。Claude Code Docs · Hooks reference — hooks are 'user-defined shell commands, HTTP endpoints, or LLM prompts that execute automatically at specific points in Claude Code's lifecycle'; events include PreToolUse, PostToolUse, Stop, and SessionEnd, and hooks can block tool calls, rewrite input/output, and inject context. As of 2026-07-17.hooks 由 harness 执行,不经过模型的自觉 —— 这是请求和规矩的分界,第 7 章展开。第三阶,是把犯过的错写成一条测试:从此那个错不归你拦,归测试拦。By the third time you run the same manual review, ask: which part can be codified? Claude Code offers three steps, each with a harder grip. Step one is CLAUDE.md: write 'run lint before committing' and it reads it every session — but the docs are explicit that this is context, not enforced configuration: it will very likely comply, with no guarantee.3注 3Note 3Claude Code Docs · Memory —— CLAUDE.md 与 auto memory 都是「上下文,不是强制配置」;要不论模型怎么决定都拦住某个动作,用 PreToolUse hook。截至 2026-07-17。Claude Code Docs · Memory — CLAUDE.md and auto memory are 'context, not enforced configuration'; to block an action regardless of what the model decides, use a PreToolUse hook. As of 2026-07-17. Step two is hooks: user-defined commands that execute automatically at fixed lifecycle points — PostToolUse formats after every edit, PreToolUse blocks it before it touches a dangerous path, Stop runs your check when it tries to finish and won't release until it passes.4注 4Note 4Claude Code Docs · Hooks reference —— hooks 是「用户定义的 shell 命令、HTTP 端点或 LLM 提示,在 Claude Code 生命周期的特定时点自动执行」;事件含 PreToolUse、PostToolUse、Stop、SessionEnd 等,可拦下工具调用、改写输入输出、注入上下文。截至 2026-07-17。Claude Code Docs · Hooks reference — hooks are 'user-defined shell commands, HTTP endpoints, or LLM prompts that execute automatically at specific points in Claude Code's lifecycle'; events include PreToolUse, PostToolUse, Stop, and SessionEnd, and hooks can block tool calls, rewrite input/output, and inject context. As of 2026-07-17. Hooks are executed by the harness, not by the model's goodwill — the line between a request and a rule, unpacked in chapter 7. Step three is turning an error it once made into a test: from then on, that error is caught by the suite, not by you.
它坑过你的方式How it burned you
沉到哪里Where it settles
约束力Grip
忘了你交代过的规矩Forgot a rule you stated
CLAUDE.md / SkillsCLAUDE.md / Skills
请求 —— 大概率照做,不保证Request — likely followed, not guaranteed
跳过了该跑的检查Skipped a required check
Hooks(PreToolUse / Stop)Hooks (PreToolUse / Stop)
规矩 —— harness 执行,跳不过Rule — harness-run, can't be skipped
犯过一次的逻辑错误A logic error it made once
一条测试用例One test case
回归 —— 从此自动拦截Regression — auto-caught from then on
最外圈,验收本身也离开你的手:claude -p 一条命令无人值守跑一遍检查,--output-format json 让脚本直接消费结果5注 5Note 5Claude Code Docs · Run Claude Code programmatically —— claude -p 非交互跑一次;--output-format 可选 text / json / stream-json,--json-schema 约束结构化输出,--allowedTools 限定权限;文档给出的场景是脚本与 CI/CD。截至 2026-07-17。Claude Code Docs · Run Claude Code programmatically — claude -p runs once, non-interactively; --output-format offers text / json / stream-json, --json-schema constrains structured output, --allowedTools scopes permissions; the documented scenarios are scripts and CI/CD. As of 2026-07-17.;接进 CI,每个 PR 自动过一遍审(第 8 章的地界)。你的角色从「亲自验每一件」退到「维护验收器 + 抽查」。顺带说清一个细节:Stop hook 连续拦 8 次后会放行1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17. —— 它是一道门,不是一座牢;真正的兜底,永远是测试和 CI 里那些不会松口的检查。On the outermost ring, the review itself leaves your hands: claude -p runs a check unattended in one command, --output-format json feeds the result straight to scripts5注 5Note 5Claude Code Docs · Run Claude Code programmatically —— claude -p 非交互跑一次;--output-format 可选 text / json / stream-json,--json-schema 约束结构化输出,--allowedTools 限定权限;文档给出的场景是脚本与 CI/CD。截至 2026-07-17。Claude Code Docs · Run Claude Code programmatically — claude -p runs once, non-interactively; --output-format offers text / json / stream-json, --json-schema constrains structured output, --allowedTools scopes permissions; the documented scenarios are scripts and CI/CD. As of 2026-07-17.; wire it into CI and every PR gets a pass automatically (chapter 8's territory). Your role retreats from verifying every piece to maintaining the verifier and spot-checking. One detail worth knowing: a Stop hook releases after 8 consecutive blocks1注 1Note 1Claude Code Docs · Best practices —— 验证一节:给 Claude 一个它能自己跑的检查(测试、构建、一张能对比的截图);没有检查,「看起来做完了」是唯一信号,你就是人肉验证回路。闸门从轻到重:同一条 prompt 里跑检查、/goal 让独立评估器每轮复查、Stop hook 当硬门(连拦 8 次后放行)、fresh-context 的验证 subagent 试图推翻结论。另有:要证据不要断言;新上下文审代码更好,「不会偏心它自己刚写的代码」;内置 /code-review 在 fresh subagent 里审当前 diff;被派去找碴的 reviewer 即使活是好的也会报出问题 —— 要求它只报影响正确性与既定要求的。截至 2026-07-17。Claude Code Docs · Best practices — the verification section: give Claude a check it can run (tests, a build, a screenshot to compare); without one, 'looks done' is the only signal and you are the verification loop. Gates from light to hard: run the check in the same prompt, a /goal condition re-checked by a separate evaluator every turn, a Stop hook as a hard gate (released after 8 consecutive blocks), and a fresh-context verification subagent that tries to refute the result. Also: evidence over assertions; a fresh context reviews better since it 'won't be biased toward code it just wrote'; the bundled /code-review reviews the current diff in a fresh subagent; a reviewer told to find gaps will report some even when the work is sound — tell it to flag only what affects correctness or the stated requirements. As of 2026-07-17. — it's a gate, not a jail; the real backstop is always the tests and CI checks that never relent.
— IV
一次对了不算数:接上评测Once Right Isn't Enough: Enter Evaluation.
单件活的验收到此闭环。但你迟早会发现自己在重复派同一类活 —— 每周的依赖升级、每个 PR 的安全审、每天的日志巡检。这时候该换单位了:单件问「这次对不对」,批量要问「这类活它多稳」。τ-bench 给这个直觉起了正式的名字:pass@k 量「k 次里至少对一次」,是能力上限;pass^k 量「k 次全对」,是可靠性。实测扎心:当时最强的 function calling agent(gpt-4o)任务成功率不到 50%,retail 域连续 8 次全对的 pass^8 掉到 25% 以下。6注 6Note 6Yao et al. · 「τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains」(2024,arXiv:2406.12045)—— 提出 pass^k「度量 agent 行为在多次试验下的可靠性」(k 次全部成功的概率,区别于 pass@k 的至少一次);实测当时最强的 function calling agent(gpt-4o)任务成功率不到 50%,retail 域 pass^8 低于 25%。Yao et al. · 'τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains' (2024, arXiv:2406.12045) — proposes pass^k 'to evaluate the reliability of agent behavior over multiple trials' (the probability that all k trials succeed, vs pass@k's at-least-once); finds the strongest function-calling agents of the day (gpt-4o) succeed on under 50% of tasks, with pass^8 under 25% in retail.跑对一次和次次跑对之间,隔着「你敢不敢把它设成定时任务」的全部距离。For a single job, the loop closes here. But sooner or later you'll notice you're delegating the same kind of job repeatedly — weekly dependency bumps, security review on every PR, daily log patrol. Time to change units: for one job you ask 'was this one right'; for a stream you ask 'how steady is it on this kind'. τ-bench gave that intuition its formal name: pass@k measures at-least-once-in-k — the capability ceiling; pass^k measures all-k-correct — reliability. The findings sting: the strongest function-calling agents of the day (gpt-4o) succeeded on under 50% of tasks, and in retail pass^8 — eight straight successes — fell below 25%.6注 6Note 6Yao et al. · 「τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains」(2024,arXiv:2406.12045)—— 提出 pass^k「度量 agent 行为在多次试验下的可靠性」(k 次全部成功的概率,区别于 pass@k 的至少一次);实测当时最强的 function calling agent(gpt-4o)任务成功率不到 50%,retail 域 pass^8 低于 25%。Yao et al. · 'τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains' (2024, arXiv:2406.12045) — proposes pass^k 'to evaluate the reliability of agent behavior over multiple trials' (the probability that all k trials succeed, vs pass@k's at-least-once); finds the strongest function-calling agents of the day (gpt-4o) succeed on under 50% of tasks, with pass^8 under 25% in retail. Between succeeding once and succeeding every time lies the whole distance of whether you dare put it on a schedule.第二个坑在「改进」上。你换了模型、改了 CLAUDE.md、加了个 skill,感觉变好了 —— 感觉不算数。评测是实验,实验自带噪音:报成功率要带标准误;比较改动前后,用同一批任务的逐题配对差,而不是两个总分相减;想在 80% 功效下检出 3 个点的差距,大约要 1000 道题。7注 7Note 7Miller · 「Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations」(Anthropic,2024-11,arXiv:2411.00640)—— 把评测当实验:报成功率要带标准误;比较两个配置,用同一批题上的题级配对差做推断,而不是两个总分相减;动手前先算最小可检测效应(MDE)—— 论文算例:80% 功效下检出 3 个点的差距约需 1000 题。Miller · 'Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations' (Anthropic, 2024-11, arXiv:2411.00640) — treat evals as experiments: report success rates with standard errors; compare two configurations via inference on question-level paired differences over the same questions, not by subtracting two totals; compute the Minimum Detectable Effect first — the paper's worked example needs roughly 1,000 questions to detect a 3-point difference at 80% power.日常尺度记一条就够:小样本上的个位数点差,不构成「变好了」的结论 —— 它只构成「再跑几遍」的理由。The second trap is 'improvement'. You switched models, edited CLAUDE.md, added a skill, and it feels better — feelings don't count. An eval is an experiment, and experiments carry noise: report success rates with standard errors; compare before and after via per-task paired differences on the same batch, not by subtracting two totals; detecting a 3-point difference at 80% power takes roughly a thousand questions.7注 7Note 7Miller · 「Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations」(Anthropic,2024-11,arXiv:2411.00640)—— 把评测当实验:报成功率要带标准误;比较两个配置,用同一批题上的题级配对差做推断,而不是两个总分相减;动手前先算最小可检测效应(MDE)—— 论文算例:80% 功效下检出 3 个点的差距约需 1000 题。Miller · 'Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations' (Anthropic, 2024-11, arXiv:2411.00640) — treat evals as experiments: report success rates with standard errors; compare two configurations via inference on question-level paired differences over the same questions, not by subtracting two totals; compute the Minimum Detectable Effect first — the paper's worked example needs roughly 1,000 questions to detect a 3-point difference at 80% power. At everyday scale, one line suffices: single-digit point differences on small samples are not a conclusion that it improved — only a reason to run it a few more times.落地不用自建 harness。把它坑过你的每个案例攒下来 —— 十几二十条,每条带上第 I 节那样的可检验标准,就是你的回归集;每次动配置,用 claude -p 把这批活重跑一遍、拿 JSON 结果对一对。5注 5Note 5Claude Code Docs · Run Claude Code programmatically —— claude -p 非交互跑一次;--output-format 可选 text / json / stream-json,--json-schema 约束结构化输出,--allowedTools 限定权限;文档给出的场景是脚本与 CI/CD。截至 2026-07-17。Claude Code Docs · Run Claude Code programmatically — claude -p runs once, non-interactively; --output-format offers text / json / stream-json, --json-schema constrains structured output, --allowedTools scopes permissions; the documented scenarios are scripts and CI/CD. As of 2026-07-17.这正是《Agent 实战》第 10 章那套 golden set + 回归的家用版,也是两本书真正合流的地方:那本书教你把评测建成产线,这本书把它接到你每天派活的台子上 —— Claude Code 是你派活的地方,评测是你确认没被坑的地方。Landing this needs no bespoke harness. Save every case where it burned you — fifteen or twenty, each with a checkable criterion like section I's — and that's your regression set; whenever you touch the configuration, rerun the batch with claude -p and diff the JSON results.5注 5Note 5Claude Code Docs · Run Claude Code programmatically —— claude -p 非交互跑一次;--output-format 可选 text / json / stream-json,--json-schema 约束结构化输出,--allowedTools 限定权限;文档给出的场景是脚本与 CI/CD。截至 2026-07-17。Claude Code Docs · Run Claude Code programmatically — claude -p runs once, non-interactively; --output-format offers text / json / stream-json, --json-schema constrains structured output, --allowedTools scopes permissions; the documented scenarios are scripts and CI/CD. As of 2026-07-17. This is the household edition of the golden-set-and-regression line in Agent in Practice, chapter 10 — and where the two books truly meet: that one teaches you to build evaluation into a production line; this one plugs it into the desk you delegate from every day. Claude Code is where you delegate; evaluation is where you make sure you weren't burned.动手 · 把这章装进你的下一次派活:Hands-on · build this chapter into your next delegation:
给你最近派出的一件活补一条能复跑的检查,并要它交证据(输出、命令、截图)—— 不收「搞定了」。Retrofit one recently delegated job with a rerunnable check, and require evidence (output, command, screenshot) — don't accept 'done'.
挑一份你本来打算直接合的 diff,先让一个全新上下文的 subagent 对着计划审一遍,数数它找出几条你没看到的。Take a diff you were about to merge as-is, have a fresh-context subagent review it against the plan first, and count what it finds that you missed.
把它最近一次坑你的错,按第 III 节的表沉淀掉:进 CLAUDE.md、变成一条 hook,或写成一条测试。Settle its most recent miss per section III's table: into CLAUDE.md, into a hook, or into a test.
派活的人已经过剩,
验收的人还稀缺.
Delegators are already plentiful;
verifiers are still scarce.
Aklman Library
— 讨论Discussion
讨论Discussion.
评论区初始化中…Initializing comments…
01 / 01
没有匹配结果No matches.
换个关键词，或按 Esc 回到页面Try another keyword, or press Esc to return