Skip to content

Stop sending dataset_name to the simulation service, and refuse it on /v2 - #4307

Draft
anth-volk wants to merge 3 commits into
masterfrom
v2plan/c1-29-no-dataset-name
Draft

anth-volk wants to merge 3 commits into
masterfrom
v2plan/c1-29-no-dataset-name

Conversation

@anth-volk

Copy link
Copy Markdown
Collaborator

Fixes #4306

Summary

  • The simulation-service client (policyengine_api/libs/simulation_reports.py) no longer sends a dataset_name key on economy simulation, economy report or budget-window requests. Household reports and re-runs never sent one.
  • POST /v2/reports (economy and budget-window kinds) and POST /v2/simulations no longer take dataset_name: it leaves EconomySimulationFields and ReportDefinitionFields, so their strict request models refuse it, as a value or as null, with 422 before any service is called.
  • Reads are unchanged: GET /v2/reports and GET /v2/simulations still name the dataset a stored report or simulation used.
  • Docstrings, the create routes' 503 descriptions and the caller-row refusal codes' comment now say the caller names a region or dynamic, and the dataset is the region's.
  • Changelog fragment: changelog.d/c1-29-no-dataset-name.removed.md.

Order of the dataset_name removal

The simulation service (PolicyEngine/policyengine-sim-api) is dropping dataset_name from its EconomyReportRequest, EconomySimulationRequest and BudgetWindowReportRequest, which refuse unknown fields. The agreed order:

  1. This PR merges and deploys to production. It stacks on nothing and needs nothing deployed first: today's simulation service declares dataset_name: … | None = None, so a body without the key is valid before and after its change.
  2. Then the simulation service's change (data_source: "local_area_dataset" for UK-wide and nation reports, and the outright removal of dataset_name) merges and deploys. Until this PR is in production there, that change must not deploy, or every create this Public API sends with the key would be refused with 422.

A later Public API change passes data_source through POST /v2/reports.

Testing

Each change was test-first: the failing tests were committed first (76736d048), seen failing for the expected reasons (the bodies still carried "dataset_name": None, and the routes answered 201), then passing. Run by node ID only:

  • tests/unit/libs/test_simulation_reports.py: test_a_report_is_posted_and_the_created_report_returned, test_a_report_sends_its_options_and_spm_settings_only_when_given, test_a_simulation_is_posted_and_the_created_simulation_returned, test_a_row_the_caller_named_that_the_service_cannot_find_is_refused (comment only).
  • tests/unit/libs/test_simulation_budget_window_reports.py: test_a_budget_window_is_posted_to_reports_and_its_window_run_returned[201] and [202], test_a_budget_window_sends_its_spm_settings_only_when_given.
  • tests/unit/v2/test_report_create_routes.py: test_a_malformed_report_is_refused_before_the_service[names-a-dataset] and [null-dataset], test_a_malformed_simulation_is_refused_before_the_service[names-a-dataset] and [null-dataset], test_a_simulation_is_created_and_answered_as_stored.
  • tests/unit/v2/test_budget_window_report_create_routes.py: test_a_budget_window_the_tables_cannot_hold_is_refused[names-a-dataset] and [null-dataset].

make quality (ruff, ruff format, Import Linter, mypy, the quality guards) passes on each commit. One read-only adversarial review round found no bugs; its documentation findings are fixed in f9f8e23b5.

No schema, workflow, Dockerfile or environment-variable change.

🤖 Generated with Claude Code

anth-volk and others added 3 commits November 14, 2026 02:19
The Public API's report, budget-window and simulation requests to the
simulation service send no dataset_name, and POST /v2/reports and
POST /v2/simulations refuse the field, given or null.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The simulation service is dropping dataset_name from its report,
budget-window and simulation requests, and it refuses unknown fields, so
the Public API stops sending the field first. POST /v2/reports and
POST /v2/simulations stop taking it too: their request models no longer
inherit it, so a body naming it, even as null, is refused with 422. Reads
still name the dataset a report or simulation used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 1's documentation findings: a creation no longer names a
dataset, so the client's module text, the create routes' 503 descriptions,
the caller-row refusal codes' comment and a test's comment say the
caller names a region or dynamic, and the dataset is the region's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.45%. Comparing base (f9c619d) to head (f9f8e23).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4307      +/-   ##
==========================================
- Coverage   93.47%   93.45%   -0.02%     
==========================================
  Files         369      369              
  Lines       21442    21453      +11     
  Branches     3400     3400              
==========================================
+ Hits        20043    20049       +6     
- Misses        816      822       +6     
+ Partials      583      582       -1     
Flag Coverage Δ
integration 56.61% <ø> (+0.02%) ⬆️
unit 91.01% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stop sending dataset_name to the simulation service, and refuse it on /v2/reports and /v2/simulations

1 participant