Skip to content

[Relax][ONNX] Add LpNormalization converter - #20818

Merged
mshr-h merged 3 commits into
apache:mainfrom
hiyufan:feat/onnx-lp-normalization-20124
Nov 17, 2026
Merged

mshr-h merged 3 commits into
apache:mainfrom
hiyufan:feat/onnx-lp-normalization-20124

Conversation

@hiyufan

@hiyufan hiyufan commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

The Relax ONNX frontend rejects checker-valid LpNormalization models because no converter is registered.

Compose existing Relax operators for L1 and L2 normalization. Honor the default axis=-1 and p=2, use absolute values for L1, and return zero for zero-norm slices. Float16/bfloat16 inputs accumulate and divide in float32 before casting back, avoiding float16 overflow. Other input dtypes are preserved. Reject unsupported values of p.

Related to #20124 (LpNormalization only).

Validation against main f0267a0a3e, Python 3.11, LLVM 21.1.8, and the CI-pinned ONNX 1.20.1 / ONNX Runtime 1.23.2:

  • 150 passed across LpNormalization, InstanceNormalization and MeanVarianceNormalization.
  • The matrix covers opsets 13/22, explicit/default p and axes, float16/float32/float64, static/symbolic shapes, negative values and zero-norm slices. All TVM cases check values and output dtypes against NumPy. ONNX Runtime checks use a separate, equivalent opset 13 model because CI's runtime lacks the opset 22 kernel; TVM's opset 22 coverage remains enabled.
  • Two additional bfloat16 VM cases use an independent NumPy reference; two cases check invalid p.
  • File-scoped pre-commit checks passed.
  • Review follow-up: explicit intermediate bb.emit bindings; 16 local L1/L2 imports across four dtypes and static/symbolic shapes remain well formed and structurally equal to the earlier version after CanonicalizeBindings.

The original CPU Jenkins failures were in the opset 22 ONNX Runtime oracle. The reference model now uses its supported opset 13 kernel without changing the model tested by TVM or skipping cases.

Prepared with OpenAI Codex assistance. The LLVM build and validation above were executed locally using the pinned TIRx/FFI sources.

The previous revision passed all eight CI checks. Fresh CI results for the review follow-up are pending.

@hiyufan
hiyufan marked this pull request as ready for review October 25, 2026 11:27
@hiyufan
hiyufan force-pushed the feat/onnx-lp-normalization-20124 branch from 486e344 to 1198e63 Compare November 15, 2026 02:51
@hiyufan

hiyufan commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@mshr-h, would you have time to review this ONNX converter, or advise how to consolidate it with the overlapping #20973?

I've rebased onto current main and fixed the CPU CI oracle failure: TVM still runs all opset 13/22 cases, while the separate ONNX Runtime reference model uses its supported opset 13 kernel. The 150 related tests pass with CI's pinned ONNX/ONNX Runtime versions, including float16/bfloat16 promotion and zero-norm handling. Fresh CI results are pending. Thank you.

@hiyufan
hiyufan force-pushed the feat/onnx-lp-normalization-20124 branch from e23fa89 to 31192cd Compare November 15, 2026 06:30
@mshr-h
mshr-h requested a max review from Copilot November 16, 2026 06:28
@mshr-h

mshr-h commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

you're earlier so lets work on yours.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

4 open issues
What changed in this PR

Adds Relax ONNX frontend support for the LpNormalization operator so checker-valid ONNX models can be imported and executed correctly.

Changes:

  • Register a new ONNX converter for LpNormalization in the Relax ONNX frontend.
  • Implement L1/L2 normalization lowering in Relax, including fp16/bf16 accumulation in fp32.
  • Add a comprehensive ONNX frontend test matrix for LpNormalization, including invalid-p and bf16 cases.
File Description
tests/​python/​relax/​test_frontend_onnx.py Adds parameterized tests covering opsets, axes, norms (p), dtypes, symbolic shapes, and error handling for LpNormalization.
python/​tvm/​relax/​frontend/​onnx/​onnx_frontend.py Introduces and registers an LpNormalization converter implemented via existing Relax ops.

Review effort: Lite

@mshr-h

mshr-h commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

please check open copilot reviews. @hiyufan

@mshr-h mshr-h self-assigned this Nov 16, 2026
@hiyufan

hiyufan commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, @mshr-h. Addressed both open Copilot threads: made intermediate bindings explicit, and verified that .ty.dtype is the correct API for the current typed-IR tree (struct_info is absent).

All 150 related tests pass with CI-pinned ONNX/ONNX Runtime, along with file-scoped pre-commit checks. The 16 local import checks also preserve the previous IR after binding canonicalization. Fresh CI results for the follow-up commit are pending.

@mshr-h
mshr-h merged commit 62a5528 into apache:main Nov 17, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants