Skip to content

hexagon: TurboVLA serves every LIBERO instruction (REPEAT on the CPU) - #85

Open
thanhdq1o2 wants to merge 1 commit into
VinRobotics:mainfrom
thanhdq1o2:hexagon-turbovla-prompts
Open

thanhdq1o2 wants to merge 1 commit into
VinRobotics:mainfrom
thanhdq1o2:hexagon-turbovla-prompts

Conversation

@thanhdq1o2

Copy link
Copy Markdown
Contributor

Summary

On the Hexagon NPU, TurboVLA aborts on any LIBERO instruction that the checkpoint pins below 21 tokens (11 or 14). For those, it fills the remaining text rows by repeating text_proj.bias [256]. The fallback wrapper sent a REPEAT to the CPU only when its source was a computed tensor, so this repeat of a weight stayed on the NPU. The NPU answered it with NO_SUPPORT, and the server aborted:

ggml-hex: HTP0 execute-op REPEAT|text_proj.bias -> node_394|256:1 -> 256:10|f32 -> f32|...
ggml-hex: HTP0 synchronize failed : dsp-error NO-SUPPORT

CPY and PAD already behave the same way when they read a weight. Now every REPEAT on Hexagon runs on the CPU. docs/backend/hexagon.md and the CHANGELOG entry are updated.

Measurements (IQ-10, main d1fe64b with and without this change)

Fixed inputs (vla_predict_check vs the CPU F32 reference): the action chunk is byte-identical with and without the change for all 13 models, GR00T N1.6 with two views, and four TurboVLA Q8_0 files. No other model repeats a weight. The other repeats (Octo, the DiT head, TurboVLA's ViT prefix) read computed tensors and already ran on the CPU.

Served through eval/client, max|Δ| of the action chunk vs a CPU-only F32 server. Each observation uses one of the four LIBERO-Object instructions:

Observation Instruction Before After
obs0 pick up the black bowl and place it on the plate 8.5e-4 8.5e-4
obs1 put the cream cheese in the bowl server aborts 3.3e-4
obs2 pick up the alphabet soup and place it in the basket server aborts 6.3e-4
obs3 open the top drawer and put the bowl inside server aborts 6.8e-4
obs4 pick up the black bowl and place it on the plate 6.3e-4 6.3e-4

Latency (vla-bench --images 2 --size 256 --warmup 3 --reps 20, the process with the lowest mean of three): 191.1 ms before, 191.4 ms after. The benchmark's prompt fills all 21 rows, so its graph does not change.

Host x86 build and ctest pass (11/11).

TurboVLA fills its 21 text rows with text_proj.bias after an instruction
the checkpoint pins below 21 tokens (11 or 14). The fallback sent a
REPEAT to the CPU only when it read a computed tensor, so this repeat of
a weight ran on the NPU, which answered NO_SUPPORT, and the server
aborted on three of the four LIBERO-Object instructions. CPY and PAD
already go to the CPU when they read a weight.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant