Skip to content

docs: pick a Q8_0 configuration per model on the IQ-10 - #84

Open
thanhdq1o2 wants to merge 4 commits into
VinRobotics:mainfrom
thanhdq1o2:hexagon-q8-recipes
Open

thanhdq1o2 wants to merge 4 commits into
VinRobotics:mainfrom
thanhdq1o2:hexagon-q8-recipes

Conversation

@thanhdq1o2

@thanhdq1o2 thanhdq1o2 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

What

  • docs/benchmark/qualcomm-iq10.md, "Q8_0 weights":
    • every Q8_0 configuration runs six checks against the CPU-only build at F32: fixed inputs and five served LIBERO observations (obs0 to obs4);
    • --keep-float lists searched to pass all six checks;
    • the configuration per model, with its quantize_gguf.py flags;
    • per model, a table of every configuration with its six checks and a table of every search with its outcome, in place of the single 62-row table;
    • levels named by what they put in Q8_0: base, base + vision, base + vision + DiT.
  • docs/backend/hexagon.md: the Q8_0 summary and example follow the six-check results.
  • CHANGELOG.md.

Why

Lists picked on one observation fail on others: OpenVLA-OFT's obs0 list passes obs0 (2.5e-3) and fails obs1 (4.3e-3). The section now reports each configuration on six checks and picks, per model, the fastest one that passes all of them and has a recorded --keep-float list.

Results on the IQ-10 at 71cdb09, mean latency and System RAM under the TEMPLATE protocol.

  • base puts in Q8_0 the 2-D weights outside the vision tower and the action head; base + vision adds the vision tower; n kept F16 keeps n of those weights out of Q8_0 with --keep-float.
  • Weights in Q8_0 counts the weights in Q8_0 out of those the level can put in Q8_0.
  • For each check, max|Δ| divided by its threshold (2.9e-3, or 1.1 times F16 where F16 is above 2.9e-3) gives a ratio; a configuration's score is its largest ratio, and it passes all checks at 1 or less.
  • A model fails when no configuration passes; its row gives the best score its searches reached, or, where no search ran, the score with all of base in Q8_0.
Model Q8_0 result Q8_0 configuration Weights in Q8_0 F16 ms Q8_0 ms vs F16 F16 System RAM MiB Q8_0 System RAM MiB vs F16
π0 passes all checks base + vision, 8 kept F16 281 of 289 1123.3 937.7 -16.5% 5662 3326 -41.3%
π0.5 passes all checks base + vision, 181 kept F16 108 of 289 1107.8 1061.2 -4.2% 5980 5095 -14.8%
Octo-Small passes all checks base + vision, 4 kept F16 119 of 123 84.1 81.5 -3.1% 725 407 -43.9%
ACT passes all checks base 35 of 35 152.5 150.9 -1.0% 469 405 -13.6%
TurboVLA passes all checks base, 12 kept F16 121 of 133 190.2 189.7 -0.3% 1079 881 -18.4%
Evo-1 passes all checks base, 66 kept F16 34 of 100 1659.0 1657.4 -0.1% 1582 1494 -5.6%
OpenVLA-OFT fails: best score 1.03 - - 1125.2 - - 15266 - -
VLA-Adapter fails: best score 1.03 - - 669.2 - - 3569 - -
GR00T N1.5 fails: best score 1.23 - - 529.3 - - 3659 - -
GR00T N1.6 fails: best score 1.33 - - 682.6 - - 4929 - -
GR00T N1.7 fails: best score 1.40 - - 744.9 - - 5108 - -
SmolVLA fails: not searched, score 44 with all of base in Q8_0 - - 584.0 - - 1226 - -
VLA-JEPA fails: not searched, score 36 with all of base in Q8_0 - - 496.8 - - 4018 - -

Every configuration with its six checks, and every search, passing or not: Per model.

Verified

  • Builds clean under -Wall -Wextra (first-party code): no code change
  • ctest passes: no code change
  • Numeric output unchanged: docs only

Archs and backends tested: SmolVLA, π0, π0.5, Evo-1, GR00T N1.5 / N1.6 / N1.7, VLA-JEPA, VLA-Adapter, OpenVLA-OFT, Octo-Small, TurboVLA and ACT on the IQ-10's Hexagon NPU, against the CPU-only build (-DGGML_HEXAGON=OFF, --weight-dtype f32 --flash-attn 0).

Check every Q8_0 configuration on fixed inputs and five served LIBERO
observations, search --keep-float lists that pass all six checks, and
list the configuration per model: π0, π0.5, Octo-Small, ACT, TurboVLA
and Evo-1 have one; GR00T N1.5 / N1.6 / N1.7, VLA-JEPA, SmolVLA,
VLA-Adapter and OpenVLA-OFT stay at F16. The Q8_0 results move to one
table per model.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant