Repository navigation
Add hid_set_num_input_buffers() API - #787
Conversation
Exposes a new public function to resize the per-device input report queue. High-throughput HID devices (medical telemetry, high-poll-rate gaming peripherals, data acquisition hardware) emit bursts of input reports that exceed the current hardcoded queue sizes (30 on macOS and the libusb backend, 64 on Windows). When a burst exceeds the queue, reports are silently dropped with no error indication to the caller. This adds: - hid_set_input_report_buffer_size(dev, size) in hidapi.h - HID_API_MAX_INPUT_REPORT_BUFFER_SIZE (1024) cap to prevent unbounded memory growth Per-backend behavior: - macOS: resizes the userspace IOHIDQueue-fed report queue - Linux libusb: resizes the userspace report queue - Windows: wraps HidD_SetNumInputBuffers (parity with existing behavior) - Linux hidraw: no-op (kernel manages buffering) - NetBSD: no-op (kernel manages buffering) Defaults are unchanged, so existing callers are unaffected. Values outside [1, HID_API_MAX_INPUT_REPORT_BUFFER_SIZE] are rejected with -1. Thread-safe on macOS (dev->mutex) and libusb (dev->thread_state), matching the locks used by the respective report callbacks. Addresses the same need as closed issue libusb#154 (HidD_SetNumInputBuffers exposure) and complements libusb#725 (callback-based input API).
Youw
left a comment
Youw
left a comment
There was a problem hiding this comment.
refer to comments, otherwise looks good
Per maintainer feedback on PR libusb#787: - Remove if (!dev) validation from all 5 backends. hidapi convention is that device functions trust the caller to pass a valid handle; only hid_close is permitted to accept NULL. - Reword the inline comment in linux/hid.c and netbsd/hid.c to lead with "No-op" so the caller-visible behavior is explicit at the implementation site.
|
Thanks for reviewing. I have made the changes. |
|
I think the name of the Win32-API |
… controls the number of input report buffers, not their byte size. - Function: hid_set_input_report_buffer_size -> hid_set_num_input_buffers - Macro: HID_API_MAX_INPUT_REPORT_BUFFER_SIZE -> HID_API_MAX_NUM_INPUT_BUFFERS - Parameter: buffer_size -> num_buffers - Error string: "buffer_size out of range" -> "num_buffers out of range"
auxcorelabs
changed the title
|
Good callout @JoergAtGithub. Have renamed the function to @Youw - Please let me know if any further changes required. Thanks both. |
|
@auxcorelabs If not, do you have a simple test to share? Thanks. |
…ile commentary, add hidtest coverage - hidapi/hidapi.h: replace the Defaults per backend list with an @note Per-backend behavior block covering macOS / Windows / libusb / hidraw / uhid semantics, ranges, and defaults. Per @Youw, the public header is the canonical place for the cross-backend contract. - linux/hid.c, netbsd/hid.c: drop the comment that cross-referenced other backends. The (void)num_buffers; idiom and the header contract speak for themselves. - libusb/hid.c: drop the self-scoped no-error-registration note for the same reason. - hidtest/test.c: add a compile-time symbol reference and a runtime call hid_set_num_input_buffers(handle, 500) right after hid_open() succeeds, per @mcuee. Both guarded on HID_API_VERSION >= 0.16.0 so they activate in the 0.16 release cycle, matching the precedent of hid_send_output_report at 0.15.0.
There was a problem hiding this comment.
Pull request overview
Adds a new public HIDAPI entry point to let callers increase the per-device input report queue depth, addressing silent report drops during bursty/high-throughput input on backends that queue reports in userspace (macOS + Linux/libusb) and exposing the existing Windows kernel buffering control.
Changes:
- Introduces
hid_set_num_input_buffers(dev, num_buffers)and documents it inhidapi.h(with a max-cap macro). - Updates macOS and Linux/libusb backends to enforce queue limits via a per-device
num_input_buffersvalue (default 30). - Adds no-op stub implementations for Linux hidraw and NetBSD uhid; updates
hidtestto exercise the new API when available.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
hidapi/hidapi.h |
Declares and documents the new public API and its max-cap macro. |
mac/hid.c |
Adds per-device buffer cap field, uses it in the report callback, and provides a setter guarded by the device mutex. |
libusb/hid.c |
Adds per-device buffer cap field, uses it in the read callback, and provides a setter guarded by the thread-state mutex. |
windows/hid.c |
Implements the API by forwarding to HidD_SetNumInputBuffers. |
linux/hid.c |
Adds a validated no-op stub for the hidraw backend. |
netbsd/hid.c |
Adds a validated no-op stub for the uhid backend. |
hidtest/test.c |
Calls the new API under a version guard to ensure all backends export it. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Per-backend helper consistency:
* linux/hid.c, netbsd/hid.c, mac/hid.c: setter uses
register_device_error() instead of register_error_str() directly.
Build-time override:
* hidapi/hidapi.h: wrap HID_API_MAX_NUM_INPUT_BUFFERS in #ifndef
so downstreams can set the cap via
-DHID_API_MAX_NUM_INPUT_BUFFERS=<value>.
Ring buffer input queue:
* New static-in-header helper hidapi_input_ring_*, present in
libusb/ and mac/ as byte-identical copies.
* libusb/hid.c, mac/hid.c: replace struct input_report * linked
list with fixed-size ring. Enqueue is O(1); eviction is inline
in push; the setter shrinks via drop_oldest so
dev->num_input_buffers is the exact steady-state cap.
* ABI unchanged (hid_device is opaque in hidapi.h).
* Allocation failure in the read callback is now handled — the
previous code had an unchecked malloc() that would segfault.
libusb has no active error channel so the drop is silent there;
mac calls register_device_error.
Youw
left a comment
Youw
left a comment
There was a problem hiding this comment.
This is an impressive amount of new code contributed in a short amount of time.
What AI tool has been used to generate it? I will not believe it was hand-written.
This was a relatively large change, developed using a structured workflow with human oversight & guidance. The design and approach/choices were planned and reviewed (via AI-assisted passes and manually) before any code change. Two AI models were then used in complementary roles—one for code drafting and the other for iterative review to refine the result. The output went through human review and feedback, and the final changes were manually validated, including building across available platforms to verify there were no errors before committing. From my experience, the effectiveness comes primarily from the structured workflow and repeated AI review iterations, followed by human review & guidance. |
I generally don't mind as I'm using similar tools and aproaches myself in various other projects. But since HIDAPI is open-source well-know public repo with many users and this is a first huge contribution to HIDAPI using AI-tools, I'd like to mention this explicitly in a form suggested by https://docs.kernel.org/process/coding-assistants.html, specifically the commit message/PR description attribution (at least I'm planning to include it in the final squash-merge commit into master), e.g.: that's why I asked what AI tool/model was used to produce this. I'll make a generic NOTE about this somewhere in the README a bit later. |
Switch ring storage from pointer-slot (per-push malloc) to a single flat buffer of (dev->num_input_buffers × slot_size), allocated at device open. slot_size: kIOHIDMaxInputReportSizeKey on macOS, interrupt-IN wMaxPacketSize on libusb, 64 B fallback. Resize re-allocs and memcpys survivors FIFO-order. Pop becomes copy-out (pop_into). Ring header consolidated into core/. Adds tests/test_hidapi_input_ring.c — 21 ASAN tests.
Thanks for the information. Please find details below: |
Youw
left a comment
Youw
left a comment
There was a problem hiding this comment.
This looks great, thanks!
I did a quick review, but I want to go over it again what I have a bit more free time than now.
Thanks for the contribution! I'll have it merged some time soon.
1: Drop register_device_error() from mac/hid.c hid_report_callback (races with user-thread hid_error() per the hidapi wiki Multi-threading Notes). Bundled cleanup of three stale linked-list/report-list comments that predate the ring buffer. 2: Coalesce the ring's two allocations into a single block with a three-pointer model — r->storage owns the allocation; r->lengths and r->data are non-owning views into the lengths region and slot-data region respectively. One calloc, overflow guards on both factors and the sum, push/pop_into index via r->data. Tests: 12 CHECKs added to assert all three pointers null post-destroy. Composite && assertions split for diagnostic precision.
Thanks! |
|
@Youw - just wanted to check whether you’ve had a chance to review this. Thanks. |
|
Somehow the added test is not built under Windows, only libusb and macOS. |
@mcuee - The tests/test_hidapi_input_ring.c are unit tests, all of them exercising the ring buffer (hidapi_input_ring.h). The ring buffer only links into mac and libusb backends — Windows wraps HidD_SetNumInputBuffers natively (our hid_set_num_input_buffers is a thin wrapper there). Hence these tests were gated to only libusb and mac. Since hidtest/test.c builds everywhere, that's where symbol-availability check and a runtime smoke call to hid_set_num_input_buffers were added. |
|
I see. Thanks for the explanation. |
Add the FreeBSD hidraw setter, preserve read interruption, and size libusb ring slots for descriptor-sized input reports. Move the shared ring and top-level tests under src, updating build, packaging and documentation references. Extend queue and virtual-device regression checks. Related to libusb#154, libusb#274, libusb#728, libusb#730 and libusb#799. Assisted-by: Codex:gpt-6
Youw
added
the
ci-virtual-device
Reset setter errors consistently across backends and cover invalid-then-valid calls in the virtual-device suite. Correct internal zero-length-read comments and explain that the buffer limit is determined by the library build. Related to libusb#787 and libusb#154. Assisted-by: Codex:gpt-6 Assisted-by: Codex:gpt-6-astra Assisted-by: Claude Code:claude-fable-5-1
|
Generated by Codex (GPT-6 Astra) and Claude Code (Claude Fable 5.1). Consolidated and fixes implemented by Codex (GPT-6). Fresh independent reviews covered the complete PR at
No unresolved review findings remain. The existing decisions are preserved: one allocation at actual queue capacity, no callback-thread error writes, no new handle-validity checks, and public-header documentation of validated no-ops. Validation on final commit
macOS device tests self-skip on hosted CI; BSD coverage is build-only. |
Adds
hid_set_num_input_buffers(dev, num_buffers)to configure the per-device input report queue.HidD_SetNumInputBuffers; default is 64 reports, with the Windows range of 2–512.Counts must be in
[1, HID_API_MAX_NUM_INPUT_BUFFERS](default maximum: 1024, overridable when building the library). Windows additionally applies its native limits. Successful setter calls clear previous device errors.The shared ring in
src/core/hidapi_input_ring.huses one allocation sized to the requested capacity, with allocation-free push/pop. Full queues drop the oldest report; shrinking retains the newest reports in FIFO order. Resize and callbacks use the backend's existing queue mutex. Callback threads do not modify device error strings.Slots use the runtime input-report size: the IOKit report-size property on macOS, and the larger of the descriptor-derived report size and endpoint packet size on libusb. This preserves large-report support and read-interruption behavior from current master. Initialization failures release the libusb interface and ring storage before a retry.
All former top-level
core/andtests/content is undersrc/. Build, distribution, CI, documentation and comment references use the new paths.src/tests/test_hidapi_input_ring.ccovers FIFO, overflow, resizing, invalid arguments, large reports and stress; virtual-device I/O tests exercise queue validation and resizing. The pkg-config consumer is insrc/tests/pkg-config/.hidtestalso exercises the public API.Related to #154.
Related to #274 and #728 (large input reports).
Related to #730 (FreeBSD hidraw).
Related to #799 (read interruption).
Assisted-by: Claude Code CLI:claude-opus-4.6
Assisted-by: Claude Code CLI:claude-opus-4.7
Assisted-by: Codex CLI:gpt-5.4
Assisted-by: Codex:gpt-6
Assisted-by: Codex:gpt-6-astra
Assisted-by: Claude Code:claude-fable-5-1