Skip to content

Store a PR run's analysis as an update on its base, not a full analysis.json #170

Open

Description

@Svilen-Stefanov

Problem

Every review run uploads the head's full analysis.json, even when an incremental run changed only a few scopes. On CodeBoarding/crate#1 (run 42718797329), the head analysis is 249 MB (a 9 MB zipped artifact), and almost all of it equals the base:

same as base
components 280 of 284
relations (top-level and nested) 6,780 of 6,907
methods_index entries 37,144 of 37,272
files 3,569 of 3,577

The changed records alone come to 1.3 MB gzipped, against 9.1 MB for the full document. That's from a naive whole-record diff; a per-edge diff would be smaller still.

Every push to a pull request uploads another full copy. Each reader then diffs the two full documents again to find out what changed, for example the webview's computeCommitDiff and scripts/diff_to_mermaid.py.

Proposal

Keep one full image per commit:

  • the analysed branch's baseline on codeboarding/baseline, updated on sync as today;
  • the base artifact per merge-base commit, which is already stored once per commit.

A review run uploads only an update on top of that base, e.g. analysis.delta.json:

  • The base it applies to: the commit sha and a hash of the base analysis.json, so a reader can refuse a delta that doesn't match.
  • Changed and removed records, keyed by stable ids:
    • component_id for components;
    • (scope, src_id, dst_id) for relations;
    • the method key for methods_index;
    • the path for files.
  • The head's metadata.

Readers apply it to the base:

  • webview and graph-viewer;
  • diff_to_mermaid.py;
  • closed_features.py, which can send the base once plus the delta.

The delta also tells each reader directly what changed, so it no longer has to diff two full documents.

A full run with no base to build on keeps uploading the full document.

Open points

  • Where the delta is computed. Diffing base and head in the Action after the analysis needs no Core change and works for every language. Having Core write it from the incremental run's changed scopes is the alternative.
  • Chained runs. A PR run seeded from an earlier run of the same PR (chain_depth) should still be a delta against the merge-base analysis, not against the previous run.
  • Retention. A base artifact must outlive every review artifact that points at it.

Done when

  • A review run on crate#1 uploads under 2 MB of analysis.
  • The PR page and the review comment are unchanged.
  • Sync still writes the full image to the baseline branch.

Activity

  1. Svilen-Stefanov commented on Nov 10, 2026

    @Svilen-Stefanov
    ContributorAuthor

    Related: CodeBoarding/CodeBoarding#686 shrinks the full document itself. The two are independent: a delta helps whatever size the base is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions