Skip to content

Require happy-path scenarios and mapped KQL in post-implementation pre-mortems, Fixes AB#3824306 - #497

Merged
moumighosh merged 2 commits into
masterfrom
moumighosh-healthy-premortem-workflow-pr
Nov 17, 2026
Merged

moumighosh merged 2 commits into
masterfrom
moumighosh-healthy-premortem-workflow-pr

Conversation

@moumighosh

@moumighosh moumighosh commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Enhance the design workflow's authoring brief and required independent healthyOperationAndFailureDetection review criterion. Require the later pre-mortem to contain feature-specific happy-path and failure/degradation scenarios with stable IDs mapped to actual Kusto (KQL) draft text/query IDs, output columns/units, healthy baselines or unconfirmed targets, detection signals, and owned investigation/stop-expansion/rollback actions.
  • Define measurement integrity: customer-visible completion rather than API success alone; eligibility/exposure, attempts/completions, cancellation versus matured abandonment, latency, cohorts/denominators/windows, retry/deduplication/sampling, ingestion delay, missing telemetry, and insufficient-evidence handling for zero/low samples. Retain justified per-measure non-runtime applicability with observable alternative verification.
  • Keep full pre-mortem/scenario/query authoring after implementation completion and PBI dispatch. Design provides the authoring brief only. Preserve named human acceptance, accessibility, default-off isolation, separate approved enabled-E2E query validation and release approval; no broad production activation is required.
  • Advance checkpoints to design-feature-v12, invalidate earlier prompt outputs, and preserve master's paired Human Design Review Brief/publication behavior. Update repository/plugin design and PBI contracts consistently.

Scope and integration

Cherry-picked only source commit 2cc691d7cbea1052407339dfceb468151611e1a4 onto current master, resolving checkpoint-version/documentation conflicts while preserving paired-review behavior. Exactly seven existing files changed; no Requirements Builder, application, or nested-repository changes.

Validation

  • node --test .github\extensions\DevDesignDoc\workflow.test.mjs — 66 passed, 0 failed, 0 skipped on the final integrated content, including paired-document regressions.
  • node --check .github\extensions\DevDesignDoc\workflow.mjs — passed.
  • node --check .github\extensions\DevDesignDoc\workflow.test.mjs — passed.
  • node --check .github\extensions\DevDesignDoc\extension.mjs — passed.
  • git diff --check and git diff origin/master --check — passed.
  • Inline node filesystem/heading-anchor validation — all 5 local design-skill documentation links/anchors passed.
  • Reviewed the complete master-to-head diff and exact seven-file scope; DevDesignDoc reloaded ready.

These are mocked workflow regressions and static/documentation checks, not live model evaluation or KQL semantic/query-output validation. No paid dynamic workflow, live Kusto queries, or production activation was run.

Tracking

Fixes AB#3824306

Tracking PBI: https://dev.azure.com/IdentityDivision/Engineering/_workitems/edit/3824306

Parent Feature: https://dev.azure.com/IdentityDivision/Engineering/_workitems/edit/3797944

The PBI tracks this workflow enhancement and is linked to this PR as a GitHub Pull Request artifact. Work-item linkage does not imply review, merge, query validation, or rollout approval.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@moumighosh
moumighosh requested a review from a team as a code owner November 16, 2026 19:03
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions

Copy link
Copy Markdown

❌ Work item link check failed. Description does not contain AB#{ID}.

Click here to Learn more.

@moumighosh moumighosh changed the title Require happy-path scenarios and mapped KQL in post-implementation pre-mortems Require happy-path scenarios and mapped KQL in post-implementation pre-mortems, Fixes AB#3824306 Nov 16, 2026
@github-actions

Copy link
Copy Markdown

✅ Work item link check complete. Description contains link AB#3824306 to an Azure Boards work item.

@somalaya

Copy link
Copy Markdown
Contributor

Could we explicitly require a rollout plan in the pre-mortem content contract? Alongside the monitoring and rollback requirements, it should include proposed exposure stages/audience, observation periods, and promotion/hold criteria, with a justified alternative when a staged rollout does not apply. This would make the expected rollout guidance explicit for the authoring agent. Please carry the requirement consistently into the repository/plugin guidance and author/reviewer prompts.

@moumighosh
moumighosh merged commit 148e822 into master Nov 17, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants