A Change-Control Process for an AI Visibility Prompt Set
Collect prompt and surface changes as requests, decide them in one batch a month, and treat every accepted batch as a new version of the set. Edits are never made in place: an edited prompt is the old one retired and a new one added, with its own id. Then run a bridge period: sample the old version and the new one side by side for at least one full period, so the report can show both numbers for the overlap and the reader sees how much of any movement is the redefinition. Retired prompts keep their history, marked closed. Without this, a prompt set drifts one reasonable edit at a time until no two months measure the same thing.
Every prompt set needs changing eventually. Buyers start using a new term, a prompt turns out to name your brand, a competitor you ignored becomes the one everyone shortlists. The changes are right. Made casually, they're also the most common reason a trend line stops meaning anything.
The six change types
| Change | Handled as | Bridge period |
|---|---|---|
| Add a prompt | New id in the next version | Yes, if headline rates include it |
| Retire a prompt | Id closed, history kept | Yes: report the rate with and without it |
| Edit a prompt's wording | Old id retired, new id added | Yes |
| Add a sampled surface | New surface-register row | Report it as a separate series first |
| Vendor changes a surface | Close the register row, open a new one | Not possible: annotate the break |
| Cosmetic fix, same meaning | Still a new id, noted as cosmetic | Optional; show both for a period if in doubt |
The request form
Change request CR-__
Requested by: ______ Date: ______
Type: add / retire / edit / surface / cosmetic
Prompt or surface id: ______ New id (add/edit): ______
Reason and buyer evidence: ______ (sales call, support ticket, search data)
Names our brand? yes/no (yes = belongs in the branded set)
Affects a running experiment? yes/no (yes = defer to its stopping date)
Decision: accept / reject / defer Decided in batch: ______
Version: v__ to v__ Bridge period: ______ to ______The bridge period
For one full period after a version change, sample both versions on the same surfaces and dates. An illustrative case: in the bridge month, v3 reads 14% and v4 reads 17% on the same answers window. Three points of any apparent rise are definitional, and the report says so. After the bridge, retire v3 and carry on with v4, with the chart breaking the series at the change rather than joining it.
Good and bad reasons to change a prompt
Good: buyers no longer ask it; a new term has replaced the old one in sales calls or support tickets; it names your brand and belongs in the branded set; it can't produce a vendor name at all. Bad: 'we're never cited for it', 'the answers aren't relevant to us', 'it makes the number look worse'. The bad reasons all remove evidence you don't like, which is how a prompt set slowly becomes a measure of the prompts you win.
FAQ
Can we change prompts during an experiment? — Not the treatment or control prompts. Defer the request to the experiment's stopping date; changing them mid-run ends the experiment under its own validity rule.
What if a vendor retires the surface we sample? — Close the register row and start a new series. There's nothing to bridge, so annotate the break and say the two sides aren't comparable.
How big should a batch be? — Small. If one month's batch replaces a large share of the set, you've started a new baseline, and it's more honest to call it that.
Put this to work on your own website.
Fenn finds what your customers ask, drafts the articles and site fixes, and measures what ChatGPT, Claude, Gemini, Perplexity and Grok say about you — with every change waiting for your approval.