Skip to content

[Relax][Frontend][ONNX] Add LpNormalization operator support - #20973

Closed
vamsi8213 wants to merge 4 commits into
apache:mainfrom
vamsi8213:feat/lp-normalizaton-impl
Closed

vamsi8213 wants to merge 4 commits into
apache:mainfrom
vamsi8213:feat/lp-normalizaton-impl

Conversation

@vamsi8213

Copy link
Copy Markdown

This PR adds support for the ONNX LpNormalization operator in the Relax ONNX frontend.

LpNormalization normalizes the input along axis by its L1 or L2 norm:

p = 1:  y = x / sum(|x|, axis, keepdims=True)
p = 2:  y = x / sqrt(sum(x * x, axis, keepdims=True))

Changes

  • Added an LpNormalization converter (_impl_v1) in python/tvm/relax/frontend/onnx/onnx_frontend.py, supporting the axis (default -1) and p (default 2) attributes, and registered it in the converter map.
    • Only p=1 and p=2 are accepted, as defined by the ONNX spec (onnxruntime also rejects other values); other values raise a ValueError.
    • axis is normalized and range-checked against the input rank.
    • Slices whose norm is zero produce zeros instead of NaN, matching onnxruntime. This is done with where(norm == 0, 1, norm) rather than clamping with an epsilon, so non-zero norms are never altered and float16 inputs don't underflow to 0 / 0.
  • Added test_lp_norm in tests/python/relax/test_frontend_onnx.py:
    • structural IR checks for the L2 (default) and L1 lowerings,
    • numerical correctness against onnxruntime for p in {1, 2} across positive and negative axes,
    • zero-norm slices (float32 and float16),
    • invalid p and out-of-range axis.
  • Enabled the ONNX backend node test (test_lpnormalization_default) by adding lpnormalization to _INCLUDE_OPS in tests/python/relax/test_frontend_onnx_backend.py.

@vamsi8213

Copy link
Copy Markdown
Author

Hi @tqchen, @mshr-h

Could you please review this PR?

@vamsi8213 vamsi8213 changed the title [Relax][ONNX] Add LpNormalization operator support [Relax][Frontend][ONNX] Add LpNormalization operator support Nov 11, 2026
@mshr-h
mshr-h requested a max review from Copilot November 13, 2026 04:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical float16 precision errors in normalization leave small L2 inputs unchanged and collapse large L1/L2 inputs to zeros instead of producing normalized outputs.

1 open issue
What changed in this PR

Adds ONNX LpNormalization support to the Relax frontend, allowing imported models to normalize tensors along a selected axis using L1 or L2 norms.

Changes:

  • Implements and registers the converter with attribute validation and zero-norm handling.
  • Adds structural and numerical tests covering axes, zero slices, float16 inputs, and invalid attributes.
  • Enables the default ONNX backend conformance test.
File Description
tests/​python/​relax/​test_frontend_onnx.py Tests L1/L2 lowering, numerical results, and attribute validation.
tests/​python/​relax/​test_frontend_onnx_backend.py Enables the default LpNormalization backend test.
python/​tvm/​relax/​frontend/​onnx/​onnx_frontend.py Implements and registers the LpNormalization converter.

Review effort: Max

@mshr-h
mshr-h requested a max review from Copilot November 13, 2026 10:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

3 open issues

Review effort: Lite

@hiyufan

hiyufan commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Hi, I noticed this overlaps with #20818, which I opened on October 25 for the same operator. I've updated that PR against current main and fixed its reference-runtime compatibility: TVM still runs the full opset 13/22 matrix, while the ONNX Runtime reference uses the equivalent opset 13 kernel supported by CI's pinned runtime. The tests cover defaults, axes, static/symbolic shapes, zero-norm slices, and float16/bfloat16 accumulation in float32.

Happy to collaborate or consolidate the implementations/tests. Could the maintainers advise which PR they would prefer to continue with so we can avoid duplicating work?

@mshr-h

mshr-h commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

closing as #20818 merged

@mshr-h mshr-h closed this Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants