[anth-volk]

Fixes #1069

Same-Repository Draft PR Check

• This PR is a draft.
• This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Changelog

• This PR adds a changelog fragment, changelog.d/<name>.<type>.md, whose type is breaking, added, changed, fixed or removed: c1-29a-data-source.added.md and c1-29a-no-dataset-name.removed.md.

Stacking, and what must merge and deploy first

• Stacked on Calculate UK-wide and nation reports on the local-area dataset, with their areas and withheld codes #1068 (v2plan/c1-23-local-area-report-profile): the v2 coordinator and UK workers that calculate a report on the local-area dataset, the manifest's UK local_area_reports declaration this PR reads, and the coordinator's check that each area level has a published area or a withheld code. This PR is retargeted to main when Calculate UK-wide and nation reports on the local-area dataset, with their areas and withheld codes #1068 merges, and merges after it.
• The Public API's Stop sending dataset_name to the simulation service, and refuse it on /v2 policyengine-api#4307 must be merged and deployed to production first. The Public API's master sends "dataset_name": null on every report, budget-window and simulation request, and these request models refuse unknown fields, so deploying this PR before #4307 would make every one of those creates fail with 422.
• Agreed order for removing dataset_name: (1) #4307 stops the Public API sending it and makes /v2/reports and /v2/simulations refuse it; it deploys to production. (2) This PR removes it from sim outright. No release accepts and ignores the field in between.
• The Public API passes data_source through POST /v2/reports in a later change.

Summary

• data_source: "local_area_dataset" on EconomyReportRequest (economy reports only), left out unless given and never stored as an option: the dataset it chooses is stored with the report's simulations. Null or another value, another country, or a region other than uk and the four country/* nations: 422 at data_source, with no code.
• Where the dataset comes from: the default v2 worker's UK manifest entry (local_area_reports), checked before any database read. No declaration, or another input version: 400 local_areas_unsupported. A declaration with no dataset, meaning the release routes the areas to no dataset of its own: 422 no_local_area_dataset at ["body", "data_source"], since that release's national reports calculate both area levels. Otherwise the store reads the catalog version's local-area dataset (the one default dataset every constituency and local-authority region uses) in place of the region's default, and answers 400 local_area_dataset_mismatch unless it is the declared one. Both 400s are the service's state, which the Public API answers with 503.
• The report is an ordinary report on that dataset: its simulations, population and the coordinator's input name it, with the report's own options (cliffs included when asked). A re-run reads the stored dataset by name, so it runs on it again; the re-run route takes no data_source.
• No dataset_name on EconomyReportRequest, EconomySimulationRequest or BudgetWindowReportRequest: given or null, it is refused with 422 extra_forbidden. The persistence library's EconomyRequest.dataset_name stays for re-runs, and the catalog read refuses a request naming both a dataset and the local-area dataset. A bundle lacking the catalog dataset is always dataset_not_in_bundle.
• Staging economy check: .github/scripts/simulation-staging-economy-check.sh sends data_source: "local_area_dataset" on its current-law Wales 2026 report. Calculate UK-wide and nation reports on the local-area dataset, with their areas and withheld codes #1068's coordinator fails such a run unless each area level has a published area or a withheld code, and writes those rows in the transaction that marks the run succeeded, so the check's requirement that the run succeeds covers them. DEPLOYMENT.md and the staging_economy_check job's comment say so.
• economy_run_store passed 800 lines once merged with Calculate UK-wide and nation reports on the local-area dataset, with their areas and withheld codes #1068's merge of main, so its catalog-row reads moved unchanged to economy_catalog_rows (read_catalog_rows), with the moved names re-exported.
• Docs: the entry README ("Local areas", the request and refusal paragraphs), the /reports 400 and 422 descriptions, the persistence library's README.

Deployment code and environment variables

The script, a workflow comment and DEPLOYMENT.md change; no variable or secret is added. The script reads SIMULATION_TEST_AUTH_ISSUER, SIMULATION_TEST_AUTH_AUDIENCE, SIMULATION_TEST_AUTH_CLIENT_ID and SIMULATION_TEST_AUTH_CLIENT_SECRET, mapped in the job step's env: from the GitHub Actions environment secrets SIMULATION_ENTRYPOINT_AUTH_ISSUER, SIMULATION_ENTRYPOINT_AUTH_AUDIENCE, SIMULATION_ENTRYPOINT_TEST_AUTH_CLIENT_ID and SIMULATION_ENTRYPOINT_TEST_AUTH_CLIENT_SECRET. All four exist in beta; simulation-deploy.yml and stage12-deploy.yml call the reusable workflow with secrets: inherit and release_environment: beta, and the job runs in environment: ${{ inputs.release_environment }}. ECONOMY_CHECK_DEADLINE_SECONDS and ECONOMY_CHECK_POLL_SECONDS stay unset (1800 and 15).

Not verified (deployed state only): that staging's default v2 worker declares the local-area dataset when the check runs, which is true once #1068's release has published the manifest; that staging's catalog gives both UK area types that dataset; and that a Wales run on the dense dataset (each new UK worker downloads it, about 90 s) finishes within the check's 30 minutes.

Testing

Test-first throughout: each failing test was committed first ("(failing)" in the subject), seen failing for the expected reason, then passing. Only the tests added or modified were run, each by node ID. The PostgreSQL tests ran against a disposable PostgreSQL 17 database with the Public API's v2 chain at the pinned public-api-commit and its grants applied. The apis-integ client tests ran on a client regenerated from this branch's OpenAPI document. Sim's checks (Ruff, rules rendering, quality guards including file length, lock checks, Pyright for the entry and the persistence library) pass on each commit.

• Entry, test_local_area_submissions.py: test_the_uk_worker_fixture_calculates_on_its_default_dataset, test_the_local_area_dataset_the_worker_cannot_run_is_refused_before_any_read (2 cases), test_a_report_without_a_data_source_needs_no_declaration, test_a_release_that_routes_no_areas_has_no_local_area_dataset, test_the_report_is_calculated_on_the_local_area_dataset_the_worker_routes_to, test_a_catalog_whose_area_regions_use_another_dataset_is_refused, test_a_report_on_the_local_area_dataset_is_rerun_on_it.
• Entry, test_economy_routes.py: test_a_data_source_a_report_cannot_have_is_refused (5), test_a_uk_wide_or_nation_report_may_ask_for_the_local_area_dataset (5), test_a_request_naming_a_dataset_is_refused (6), test_a_local_area_refusal_answers_with_its_code (3), test_only_an_economy_report_takes_a_data_source (2); test_economy_rerun_routes.py::test_a_report_rerun_takes_no_data_source.
• Entry, test_openapi.py: test_economy_requests_name_no_version_and_no_record_id, test_a_report_request_is_an_economy_report_a_budget_window_or_a_household, test_the_local_area_refusals_are_in_the_error_codes.
• Entry, test_economy_submissions.py: test_a_region_dataset_the_bundle_lacks_is_the_services_state, test_the_catalog_dataset_in_the_bundle_is_the_one_calculated, test_a_stored_dataset_the_catalog_lacks_is_named_in_the_refusal, test_a_refusals_detail_is_built_from_the_request_not_the_error; test_staging_economy_check.py::test_the_check_creates_a_small_report_and_waits_for_its_run_to_succeed.
• Persistence, PostgreSQL: test_economy_run_store.py::test_a_report_on_the_local_area_dataset_is_read_on_it, ::test_a_declared_dataset_the_catalog_does_not_give_both_area_types_is_refused (3), ::test_every_side_refusal_is_raised_with_its_side, ::test_a_request_naming_a_dataset_and_the_local_area_dataset_is_refused; test_economy_run_records.py::test_the_local_area_dataset_is_the_one_both_area_types_use, ::test_a_catalog_version_without_one_dataset_for_both_area_types_has_none (3), ::test_the_local_area_dataset_may_be_the_national_default.
• apis-integ: test_economy_generated_client.py::test_a_report_request_keeps_omitted_spm_settings_omitted, ::test_a_created_report_is_read_with_its_runs, ::test_a_created_simulation_is_read_with_its_run; test_budget_window_generated_client.py::test_a_budget_window_request_is_sent_as_given, ::test_a_created_budget_window_is_read_with_its_years.

Review rounds

Three read-only adversarial review rounds, each over the change against its base at the time; none found a bug.

• Round 1 (the whole change): quality findings fixed: documentation drift, test gaps (the routes' answers for the three local-area codes, the OpenAPI enums, data_source refused on budget windows, household reports and re-runs, a re-run of a report on the local-area dataset), and the catalog read now refuses a request naming two datasets (its test failed first).
• Round 2 (round 1's fixes): documentation and two test setups fixed.
• Round 3 (round 2's fixes): one description's wording fixed.

No inline suppression is added.

🤖 Generated with Claude Code