Skip to content

Schema Validation: show failures as annotations and a job summary - #3544

Merged
julianwood merged 1 commit into
mainfrom
crew/serverless-patterns/schema-validation-annotations
Nov 15, 2026
Merged

julianwood merged 1 commit into
mainfrom
crew/serverless-patterns/schema-validation-annotations

Conversation

@julianwood

Copy link
Copy Markdown
Contributor

When Schema Validation fails, the run only shows a red cross. The reasons are buried in the job log, so contributors have to open it to find out what to fix.

This change makes _scripts/validate.js report each problem in two places when it runs in GitHub Actions:

  • An ::error annotation. Pattern file and folder problems point at the pattern's example-pattern.json. Broad IAM findings point at the template file and line. Annotations show on the run page and, for files in the diff, on the Files changed tab.
  • A table in the job summary (pattern, check, problem), or a pass line, with a link to the pattern checklist in PUBLISHING.md.

Messages are escaped the way workflow commands expect (%, CR and LF in the message, plus : and , in properties).

What stays the same: the workflow file, its contents: read permission and GH_AUTOMATION: false. The script still posts no comments in this workflow. Local runs print the same output as before, and the GH_AUTOMATION=true comment and label path is unchanged.

check-iam.js gains findPatternIam, which returns each finding's file and line. checkPatternIam returns the same strings as before.

Tests: node --test _scripts/annotate.test.js covers the escaping, the annotation format and the summary. I also ran the script with GITHUB_ACTIONS=true on a passing pattern and on a temporary broken copy (missing title, a bad language, an Action: "*" grant, a flat folder):

::error file=zz-annotate-test/example-pattern.json,title=Pattern file%3A zz-annotate-test::title: is required
::error file=zz-annotate-test/template.yaml,line=141,title=Broad IAM%3A zz-annotate-test::grants every action on every resource: Action: "*"
::error file=zz-annotate-test/example-pattern.json,title=Folder structure%3A zz-annotate-test::zz-annotate-test is a flat folder. New patterns use {family}/{language}/{framework}, e.g. sqs-lambda/python/sam. See PUBLISHING.md "Folder structure".

In GitHub Actions, validate.js now prints each problem as an ::error
workflow command (pattern file and folder problems on example-pattern.json,
IAM findings on the template file and line) and appends a summary table to
GITHUB_STEP_SUMMARY. Local runs and the GH_AUTOMATION comment path are
unchanged.
@julianwood julianwood added the migration Bulk repo change (e.g. folder migration). Skips the PatternsRotator; reviewed by maintainers. label Nov 15, 2026
@julianwood
julianwood merged commit e5a9b64 into main Nov 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

migration Bulk repo change (e.g. folder migration). Skips the PatternsRotator; reviewed by maintainers.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant