Repository navigation
docs: pick a Q8_0 configuration per model on the IQ-10 - #84
Open
thanhdq1o2 wants to merge 4 commits into
Open
thanhdq1o2 wants to merge 4 commits into
thanhdq1o2 wants to merge 4 commits into
Conversation
Check every Q8_0 configuration on fixed inputs and five served LIBERO observations, search --keep-float lists that pass all six checks, and list the configuration per model: π0, π0.5, Octo-Small, ACT, TurboVLA and Evo-1 have one; GR00T N1.5 / N1.6 / N1.7, VLA-JEPA, SmolVLA, VLA-Adapter and OpenVLA-OFT stay at F16. The Q8_0 results move to one table per model.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
docs/benchmark/qualcomm-iq10.md, "Q8_0 weights":--keep-floatlists searched to pass all six checks;quantize_gguf.pyflags;docs/backend/hexagon.md: the Q8_0 summary and example follow the six-check results.CHANGELOG.md.Why
Lists picked on one observation fail on others: OpenVLA-OFT's obs0 list passes obs0 (2.5e-3) and fails obs1 (4.3e-3). The section now reports each configuration on six checks and picks, per model, the fastest one that passes all of them and has a recorded
--keep-floatlist.Results on the IQ-10 at
71cdb09, mean latency and System RAM under the TEMPLATE protocol.--keep-float.Every configuration with its six checks, and every search, passing or not: Per model.
Verified
-Wall -Wextra(first-party code): no code changectestpasses: no code changeArchs and backends tested: SmolVLA, π0, π0.5, Evo-1, GR00T N1.5 / N1.6 / N1.7, VLA-JEPA, VLA-Adapter, OpenVLA-OFT, Octo-Small, TurboVLA and ACT on the IQ-10's Hexagon NPU, against the CPU-only build (
-DGGML_HEXAGON=OFF,--weight-dtype f32 --flash-attn 0).