Repository navigation
hexagon: TurboVLA serves every LIBERO instruction (REPEAT on the CPU) - #85
Open
thanhdq1o2 wants to merge 1 commit into
Open
thanhdq1o2 wants to merge 1 commit into
thanhdq1o2 wants to merge 1 commit into
Conversation
TurboVLA fills its 21 text rows with text_proj.bias after an instruction the checkpoint pins below 21 tokens (11 or 14). The fallback sent a REPEAT to the CPU only when it read a computed tensor, so this repeat of a weight ran on the NPU, which answered NO_SUPPORT, and the server aborted on three of the four LIBERO-Object instructions. CPY and PAD already go to the CPU when they read a weight.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On the Hexagon NPU, TurboVLA aborts on any LIBERO instruction that the checkpoint pins below 21 tokens (11 or 14). For those, it fills the remaining text rows by repeating
text_proj.bias[256]. The fallback wrapper sent aREPEATto the CPU only when its source was a computed tensor, so this repeat of a weight stayed on the NPU. The NPU answered it withNO_SUPPORT, and the server aborted:CPYandPADalready behave the same way when they read a weight. Now everyREPEATon Hexagon runs on the CPU.docs/backend/hexagon.mdand the CHANGELOG entry are updated.Measurements (IQ-10, main
d1fe64bwith and without this change)Fixed inputs (
vla_predict_checkvs the CPU F32 reference): the action chunk is byte-identical with and without the change for all 13 models, GR00T N1.6 with two views, and four TurboVLA Q8_0 files. No other model repeats a weight. The other repeats (Octo, the DiT head, TurboVLA's ViT prefix) read computed tensors and already ran on the CPU.Served through
eval/client, max|Δ| of the action chunk vs a CPU-only F32 server. Each observation uses one of the four LIBERO-Object instructions:Latency (
vla-bench --images 2 --size 256 --warmup 3 --reps 20, the process with the lowest mean of three): 191.1 ms before, 191.4 ms after. The benchmark's prompt fills all 21 rows, so its graph does not change.Host x86 build and
ctestpass (11/11).