How to Review an SEO Agent Pull Request Before Merging

A human PR gets line-by-line review because humans err line by line. An agent PR encodes one decision applied many times — so the review target is the decision, plus enough sampling to catch where the rule met an exception it didn't understand.

The three-pass method

1. Pattern pass: reconstruct the rule from the diff ('titles under 30 chars got the product name appended'). If you can't state the rule, the agent may not have one — stop there
2. Sample pass: random 5-10 changed pages, checked by hand against the rule and against sense; randomness matters because agent errors cluster in the tail cases, not the showcase cases
3. Semantic pass: the checks no linter runs — new titles still true of their pages, canonical targets exist and are the right canonical, schema edits still match what a reader sees

Worked example

A PR retitles 38 pages and repoints 12 canonicals. Pattern pass: the rule is 'append category to sub-40-char titles; point paginated duplicates at page one' — sane. Sample pass: 8 random pages; 7 fine, one appended a category to a legal page where it reads absurd — a tail case the rule missed. Semantic pass: 2 of the 12 canonical targets 308 to other URLs, so the canonical should point at the redirect target, not the redirecting URL. Verdict: request changes on the legal-page exclusion and the two canonical targets; the other 46 edits are approved as a class.

A failure worth checking

The clean-diff trap: agent PRs are tidy — consistent formatting, plausible commit messages — and tidiness reads as correctness to a tired reviewer. The two canonical-to-redirect errors in the example above produce a perfectly clean diff. Neatness is how the error dresses; the sample pass is how you undress it.