Repository navigation

docs: record the backup state and restore rehearsal of the beta database #1597

Workflow file for this run

	name: Review

	# A second pair of eyes, enforced rather than hoped for.
	#
	# WHY THIS EXISTS. Phase 2 of the lean transformation removed ~24,000 lines
	# straight to `main` with nothing reviewing it. That was recorded all along as an
	# accepted risk — "one developer, app is stopped" — but the recording was wrong
	# about the cause: branch protection existed the whole time with the right
	# required checks, and `enforce_admins` was simply false, so every push walked
	# past it. The risk was a misconfiguration nobody checked, not a decision.
	#
	# WHAT IT MUST CATCH, and why a generic code review will not. The failure mode of
	# a deletion program is not a bug; it is deleting a control while it looks like
	# deleting ceremony. Phase 2 deleted, correctly, a signed approval bundle, two
	# cache-generation comparisons and an approval-binding FK — and kept, correctly,
	# a live kill switch, a permit state machine and a spend cap. Those decisions
	# look identical in a diff. Telling them apart means reading the tree to find
	# what still enforces the rule, which is why this uses Claude Code rather than a
	# diff-only reviewer: it can grep.
	#
	# PRECISION OVER RECALL ON THE BLOCKING PATH. A CONTROL_REMOVAL or UNEVIDENCED
	# finding fails the job. NOTE findings do not. A reviewer that flags everything
	# trains you to click past it, which is a rubber stamp with extra steps — the
	# same reasoning as the Fallow audit's `gate: new-only`.
	#
	# NOT A BOT APPROVAL. GitHub counts a GitHub App review toward
	# `required_approving_review_count`. This deliberately does not do that: it
	# reports findings and exits non-zero. A red required check is honest; a bot
	# approving a bot's work satisfies the rule and protects nothing.
	#
	# CREDENTIALS. Either works, and the jobs accept whichever is present:
	# - `CLAUDE_CODE_OAUTH_TOKEN` — what `/install-github-app` provisions. In use.
	# - `ANTHROPIC_API_KEY` — a static API key.
	# Best option if you have Anthropic org admin: Workload Identity Federation
	# (`anthropic_federation_rule_id` + `anthropic_organization_id`, with the
	# `id-token: write` already granted below), which exchanges the workflow's
	# GitHub OIDC token for a short-lived credential and leaves no static secret.
	#
	# THIS FILE CANNOT REVIEW ITS OWN CHANGES. claude-code-action refuses to run
	# when the workflow file differs from the copy on the default branch:
	#
	# Skipping action due to workflow validation: the workflow file must exist
	# and have identical content to the version on the repository's default branch
	#
	# That is a real control, not a bug — otherwise a pull request could rewrite its
	# own reviewer to print secrets. The consequences are worth knowing:
	# - Until this file is merged to `main`, both reviewers skip on every run.
	# - Any future PR that edits this file will skip them again, and must be
	# reviewed by a human.
	# - A skipped action exits SUCCESS and emits nothing. Both jobs therefore fail
	# when their verdict file is absent. That is not defensive programming: the
	# security job actually reported green through four runs in which nothing
	# was reviewed, which is how the requirement was found.
	#
	# NOT A REQUIRED CHECK, BY DECISION (2026-09-28): `security` and
	# `control-vs-ceremony` stay informational (docs/operations/security-ci-policy.md,
	# "The AI review workflow is not a gate"). A bot approving a bot's work protects
	# nothing, and a lapsed credential must never block every merge.

	on:
	pull_request:
	types: [opened, synchronize, reopened]

	concurrency:
	group: ${{ github.workflow }}-${{ github.ref }}
	cancel-in-progress: true

	permissions:
	contents: read

	jobs:
	# Fail explicitly when no credential is present, rather than letting the
	# review jobs skip and leave a green-looking PR that nothing examined.
	credentials:
	runs-on: ubuntu-latest
	steps:
	- name: Require review credentials
	env:
	OAUTH: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
	API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
	run: \|
	if [ -z "${OAUTH}" ] && [ -z "${API_KEY}" ]; then
	echo "::error::No review credential is set. The review gate cannot run."
	echo "::error::Add CLAUDE_CODE_OAUTH_TOKEN (what /install-github-app"
	echo "::error::provisions) or ANTHROPIC_API_KEY under Settings >"
	echo "::error::Secrets and variables > Actions."
	exit 1
	fi

	control-vs-ceremony:
	needs: [credentials]
	runs-on: ubuntu-latest
	permissions:
	contents: read
	pull-requests: write
	# The action exchanges the workflow's GitHub OIDC token for its App token;
	# without this it fails with "Failed to get OIDC token" before it ever
	# reaches the model. Also what Workload Identity Federation needs.
	id-token: write
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	# Full history so the reviewer can diff against the base and read what
	# a deleted line used to do, not merely that it is gone.
	fetch-depth: 0

	- name: Adversarial control-vs-ceremony review
	id: review
	uses: anthropics/claude-code-action@db77d4d3c98f3f6380e7cd45ffba2342a3dbde3c # v1
	with:
	claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
	anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
	prompt: \|
	You are an adversarial reviewer on a codebase mid-way through a
	deliberate deletion program. Roughly 24,000 lines have already been
	removed on purpose. Your job is NOT to object to deletion.

	Your job is to find the ONE deletion that removed a CONTROL while
	looking like it removed CEREMONY.

	The distinction this codebase uses:
	- CEREMONY: machinery whose only job is to prove a document, a
	catalogue, a counter or a snapshot is consistent with itself.
	Deleting it is the point of this program.
	- CONTROL: anything that changes what the software will actually
	refuse to do. A live authorization check, a spend cap, a kill
	switch, a state machine, a tenant predicate, a deadline, an input
	bound, a fail-closed branch.

	For EVERY removed or weakened check in this diff, answer all four:
	1. What did this gate before?
	2. What gates it now? Name the file and the line.
	3. Is that replacement evaluated LIVE per request, or is it a
	stored snapshot compared later? A snapshot is not equivalent
	to a live read.
	4. If it was a test: did it lose its SUBJECT (the behaviour no
	longer exists, so deletion is correct), or did it lose
	COVERAGE (the behaviour still exists and is now unasserted,
	which is a finding)?

	Read the tree to answer 2 and 3. Do not answer from the diff alone;
	a diff cannot show you what still enforces a rule. Use grep.

	Traps this codebase has already sprung, so look for them:
	- An edit reported as applied that silently did not apply. Verify
	claims in the commit message against actual file contents.
	- A test that passes vacuously because its input was mangled before
	assertion. If a test searches text, check the text survived.
	- A required field dropped from a type while a SQL function, a
	regprocedure cast, a GRANT or a placeholder list still names it.
	- A deny code removed from a union while a downstream mapper still
	classifies it.
	- A guard whose parameter is now unused, which usually means the
	guard stopped guarding before it stopped existing.
	- A workflow or config whose subject was deleted, leaving a required
	check that cannot parse its own input.

	OUTPUT. Write your findings to `verdict.json` in the repository
	root, and write nothing else. Shape:

	{"verdict":"BLOCK"\|"PASS",
	"findings":[{"classification":"CONTROL_REMOVAL"\|"UNEVIDENCED"\|"NOTE",
	"summary":"...","evidence":"path:line"}]}

	Set `verdict` to BLOCK if and only if there is at least one
	CONTROL_REMOVAL or UNEVIDENCED finding; NOTE findings alone must
	produce PASS. Every finding needs `evidence` naming a file and line
	— a finding you cannot point at is a NOTE at most.

	Do not hedge. An uncertain BLOCK is more useful than a confident
	PASS, but a reflexive BLOCK on every pull request makes this gate
	worthless, so justify each one.
	# A FILE, not `--json-schema`. The schema route failed with "Claude
	# did not return structured_output", and the run behind that message
	# had num_turns 1, total_cost_usd 0 and empty modelUsage after 1.9s —
	# the CLI died before reaching the model, and the action reports the
	# missing output rather than the cause. A file the model writes with a
	# tool it demonstrably has is one less moving part, and the dump step
	# below now surfaces the underlying error either way.
	claude_args: \|
	--allowed-tools Read,Grep,Glob,Write,Bash(git diff:),Bash(git log:),Bash(git show:*)

	# Prints why the reviewer failed instead of leaving it in a file on a
	# discarded runner. Added because two full cycles were spent guessing at a
	# message that described a symptom.
	- name: Dump the reviewer execution log
	if: always()
	env:
	EXEC: ${{ steps.review.outputs.execution_file }}
	run: \|
	echo "conclusion: ${{ steps.review.outputs.conclusion }}"
	if [ -n "${EXEC:-}" ] && [ -f "${EXEC}" ]; then
	echo "--- execution log ---"
	tail -c 4000 "${EXEC}"
	else
	echo "no execution file at '${EXEC:-<unset>}'"
	fi

	- name: Enforce the verdict
	env:
	GH_TOKEN: ${{ github.token }}
	PR: ${{ github.event.pull_request.number }}
	REPO: ${{ github.repository }}
	run: \|
	set -euo pipefail
	# Read from the file the reviewer wrote. Two earlier designs failed
	# here for reasons worth keeping:
	# 1. Scraping "REVIEW-VERDICT:" off the last line of a PR comment.
	# The action posts a pull-request REVIEW, not an issue comment,
	# so the scraper found nothing. A gate that parses prose fails on
	# formatting.
	# 2. `--json-schema` + `structured_output`. The action reported the
	# output missing while the real failure was upstream of the model
	# (one turn, zero cost, 1.9s).
	# Absent or unparseable file is still a failure: a review gate with no
	# verdict has not reviewed anything.
	if [ ! -f verdict.json ]; then
	echo "::error::The reviewer wrote no verdict.json."
	echo "::error::See the execution log dumped by the previous step."
	exit 1
	fi
	if ! jq -e '.verdict and (.findings \| type == "array")' verdict.json >/dev/null 2>&1; then
	echo "::error::verdict.json is not the required shape:"
	head -c 2000 verdict.json
	exit 1
	fi
	verdict=$(jq -r '.verdict' verdict.json)
	echo "verdict: ${verdict}"

	# Post the findings so the reasoning lands where a reader sees it,
	# not only in a job log.
	{
	echo "## Control-vs-ceremony review"
	echo
	echo "Verdict: ${verdict}"
	echo
	jq -r 'if (.findings \| length) == 0 then "No findings." else (.findings[] \| "- \(.classification) — \(.summary)\n - evidence: \(.evidence)") end' verdict.json
	} > review-comment.md
	gh pr comment "${PR}" --repo "${REPO}" --body-file review-comment.md

	# Belt and braces: block on the verdict OR on any blocking finding, so
	# a model that mislabels its verdict while correctly classifying a
	# finding still stops the merge.
	blocking=$(jq '[.findings[] \| select(.classification == "CONTROL_REMOVAL" or .classification == "UNEVIDENCED")] \| length' verdict.json)
	if [ "${verdict}" = "BLOCK" ] \|\| [ "${blocking}" -gt 0 ]; then
	echo "::error::Review found ${blocking} blocking finding(s). See the PR comment."
	exit 1
	fi

	security:
	needs: [credentials]
	runs-on: ubuntu-latest
	permissions:
	contents: read
	pull-requests: write
	id-token: write
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	with:
	fetch-depth: 0

	# Kept separate from the reviewer above on purpose: that one asks "did a
	# control disappear", this one asks "is there a vulnerability class here".
	# Merging them dilutes both prompts.
	#
	# Uses the same pinned action rather than
	# `anthropics/claude-code-security-review`, which publishes no tags.
	# `check:action-pins` requires a SHA plus a `# v…` comment and there is no
	# version to name — so the options were to widen a security policy for one
	# convenience, or not depend on an unversioned action. This program exists
	# to stop making the first trade.
	- name: Security review
	id: review
	uses: anthropics/claude-code-action@db77d4d3c98f3f6380e7cd45ffba2342a3dbde3c # v1
	with:
	claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
	anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
	prompt: \|
	Review this pull request for security vulnerabilities only. Ignore
	style, naming and architecture — another reviewer covers those.

	This codebase handles merchant OAuth credentials for Google Business
	Profile and calls a paid AI provider, so weight these classes:
	- credential exposure: a token, client secret or encryption key
	reaching a log, an error body, a URL, an event payload, a
	persisted column or the browser
	- authorization bypass: a path reaching a provider or a tenant's
	data without re-proving organization, property and capability
	- tenant isolation: a query missing its organization predicate
	- spend: an AI provider call that can be made without passing the
	per-route cost ceiling
	- injection: SQL built by concatenation, a shell argument from
	request data, a regex from user input
	- fail-open: a catch that swallows a refusal, a default that
	permits, an optional dependency whose absence skips a check

	Report only findings you can evidence with a file and line. For
	each, state the input that reaches it and what an attacker gains.
	Do not report theoretical issues with no reachable path — a false
	positive here costs more than it saves, because it teaches the next
	reader to skim.

	OUTPUT. Write your findings to `security.json` in the repository
	root, and write nothing else. Shape:

	{"findings":[{"severity":"HIGH"\|"MEDIUM"\|"LOW","summary":"...",
	"evidence":"path:line","attacker_gains":"..."}]}

	Write the file even when you find nothing — an empty `findings`
	array is the clean result, and its presence is how this gate tells
	"found nothing" apart from "did not run".
	claude_args: \|
	--allowed-tools Read,Grep,Glob,Write,Bash(git diff:),Bash(git log:),Bash(git show:*)

	- name: Dump the reviewer execution log
	if: always()
	env:
	EXEC: ${{ steps.review.outputs.execution_file }}
	run: \|
	echo "conclusion: ${{ steps.review.outputs.conclusion }}"
	if [ -n "${EXEC:-}" ] && [ -f "${EXEC}" ]; then
	echo "--- execution log ---"
	tail -c 4000 "${EXEC}"
	else
	echo "no execution file at '${EXEC:-<unset>}'"
	fi

	# Without this the job is a FALSE GREEN, and it already was one. When
	# `review.yml` differs from the copy on the default branch the action
	# skips itself by design (see the header), exits success, and produces no
	# output — so this job reported green through four runs in which no
	# security review happened at all. A gate that cannot tell "found nothing"
	# from "did not run" is not a gate.
	- name: Enforce the security findings
	env:
	GH_TOKEN: ${{ github.token }}
	PR: ${{ github.event.pull_request.number }}
	REPO: ${{ github.repository }}
	run: \|
	set -euo pipefail
	if [ ! -f security.json ]; then
	echo "::error::The security reviewer wrote no security.json."
	echo "::error::Either it was skipped by workflow validation (does"
	echo "::error::this PR change .github/workflows/review.yml?) or it"
	echo "::error::failed — see the execution log above. Not treating"
	echo "::error::silence as a clean bill."
	exit 1
	fi
	if ! jq -e '.findings \| type == "array"' security.json >/dev/null 2>&1; then
	echo "::error::security.json is not the required shape:"
	head -c 2000 security.json
	exit 1
	fi
	{
	echo "## Security review"
	echo
	jq -r 'if (.findings \| length) == 0 then "No findings." else (.findings[] \| "- \(.severity) — \(.summary)\n - evidence: \(.evidence)\n - attacker gains: \(.attacker_gains)") end' security.json
	} > security-comment.md
	gh pr comment "${PR}" --repo "${REPO}" --body-file security-comment.md

	# HIGH and MEDIUM block. LOW is reported and does not.
	blocking=$(jq '[.findings[] \| select(.severity == "HIGH" or .severity == "MEDIUM")] \| length' security.json)
	if [ "${blocking}" -gt 0 ]; then
	echo "::error::Security review found ${blocking} HIGH/MEDIUM finding(s)."
	exit 1
	fi

Navigation Menu

Repository navigation

docs: record the backup state and restore rehearsal of the beta database #1597

Workflow file

docs: record the backup state and restore rehearsal of the beta database #1597

Uh oh!

Workflow file for this run