Skip to content

Make pool and I/O reactor startup safe under pinned creators - #2

Open
ivanmilevtues wants to merge 2 commits into
mainfrom
showcase/thread-pool-utilities
Open

ivanmilevtues wants to merge 2 commits into
mainfrom
showcase/thread-pool-utilities

Conversation

@ivanmilevtues

@ivanmilevtues ivanmilevtues commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Purpose and upstream inspiration

A larger, genuine C++ worker-startup change for the CodeBoarding architecture-diff showcase. Leave this example PR unmerged. No generated analysis is edited.

This adapts two concrete upstream cuCascade problems to this fork's older I/O implementation:

  • NVIDIA/cuCascade#240: partial pool construction could destroy joinable threads and terminate the process instead of allowing optional-cache fallback.
  • NVIDIA/cuCascade#243: pools and reactors created by a pinned host worker inherited its single CPU. The upstream report describes this in Sirius/DuckDB; no throughput improvement is claimed for this PR.
  • NVIDIA/cuCascade#204 introduced the shared thread utilities and cache completion pool present in this checkout.

Changes

  • Reuse shared thread utilities for Linux name truncation and bounded CPU-ID handling.
  • Read the process main thread's CPU mask for default pool placement. Explicit CPU lists still win; inability to read/apply affinity remains best-effort.
  • Apply pool worker names and affinity before the initializer, and wait for startup before returning.
  • Reject zero/negative widths, propagate initializer exceptions, and stop/join every started worker when initialization or construction fails.
  • Apply the same process-affinity policy to REST and io_uring workers before their work loops. Prepare names and CPU vectors on the caller, before spawning, so allocation failures remain recoverable rather than escaping a worker thread.
  • Add public-behavior tests for names, explicit/default affinity, pinned creators, startup ordering, failure cleanup, invalid widths, and actual REST-worker state at its authorization boundary.

The pool now calls compiled utilities from cucascade_io; standalone header-only users must link that implementation. In-tree pool consumers already use the I/O layer. Empty pool CPU lists intentionally change from creator-thread inheritance to process-main-thread affinity.

Concrete before/after scenarios

Scenario Before After
Process main thread allows CPUs 0–11; a creator pinned to CPU 0 starts a default pool or REST reactor New workers inherit CPU 0, serializing otherwise parallel work Workers adopt CPUs 0–11 before initialization or request handling
Caller explicitly requests CPU 2 for a pool Explicit placement applies CPU 2 still wins over the default process mask
Pool initializer throws while other workers have started Exception escapes a worker and aborts the process Constructor joins all started workers, then propagates the exception to the caller
Thread creation or startup allocation fails partway through a pool Destruction of joinable threads can terminate the process All started workers are stopped and joined before rethrowing
Pool has a long descriptive name Linux rejects the overlong name Shared utilities truncate the name to the Linux limit

These are exercised by test/exec/test_thread_pool.cpp and the real REST-worker test in test/io/rest/test_rest_footer_resolve.cpp. The CPU numbers illustrate the tested machine's mask, not hard-coded placement requirements.

Verification

  • Reproduced original failures: long worker name was ignored; a pinned creator confined the pool to CPU 0 rather than allowed CPUs 0–11; a throwing initializer and a construction-allocation failure both aborted with exit 134.
  • CMake I/O target builds with GCC 14, CUDA 12.9, warnings-as-errors and CUCASCADE_BUILD_CUDF=OFF. The supplied liburing package had incompatible generated headers; liburing 2.15 was rebuilt locally against the compiler sysroot, without changing repository dependencies.
  • 7 targeted cases / 27 assertions pass, including the real REST reactor test.
  • CPU pool tests also pass under ASan/UBSan (6 cases / 24 assertions).
  • A temporary allocation-failure sweep propagated 16 different construction failures and verified all started workers exited before the exception returned.
  • Targeted pre-commit hooks pass.
  • The full I/O suite is not green: two datasource-registry tests require kvikIO although this is a cudf-OFF build; the concurrent SigV4 presigning test then hit Catch2's redirect is already active assertion and aborted (132 passed, 3 failed at that point). No failure is hidden or fixed outside this PR's scope.
  • Wider related pool/REST/footer tests pass: 26 cases / 356 assertions. io_uring's changed source compiles, but its worker affinity has no dedicated runtime test in this checkout.

Architecture map

Live PR map. The last inspected rendering still showed the original small commit 7783e3a (+53/−13), explicitly marking expanded head 87a5eab as not analyzed. Both the expanded review and its user-requested rerun failed on hosted quota (HTTP 402, token limit reached). A larger source diff cannot repair quota or refresh a stale map.

Bounded local analysis of the same five pool/utility/reactor files, using actual CMake compiler context for both base and head, resolves a new pool-constructor → thread_util::get_process_affinity call at include/cucascade/exec/thread_pool.hpp:54. Under the current baseline component assignments this crosses Task Execution Framework → Thread Execution Utilities. This relationship was absent from the equivalent baseline sample. Reactor → Thread Execution Utilities already existed through set_thread_name; this PR changes those call sites to process-affinity discovery and worker-local setup, rather than inventing a new reactor component relationship.

Hosted baseline diagnostics show missing project headers; the default-context local run also misses these calls. Providing the real compiler context resolves them. This is local call-resolution evidence, not proof that the hosted component map displays an added edge. No generated analysis or hosted analyzer configuration was changed. The bounded run still has a liburing header diagnostic and does not replace whole-project analysis.

Truncate worker names to Linux's limit and reuse bounded CPU affinity handling. Cover observable names and affinity with CPU-only Catch2 tests.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c140-9c31-7237-bc6b-617fa475f82a
Co-authored-by: Amp <amp@ampcode.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-09T17:15:51.795921Z 7783e3a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

CodeBoarding review · failed

Your CodeBoarding quota is exhausted. Buy a CodeBoarding license at https://codeboarding.org/pricing, or wait for your weekly reset.

See the workflow logs.

Use the process main thread's CPU mask for default worker placement. Apply names and affinity before initialization and reactor work, propagate pool initializer failures, and join workers on partial startup failure. Prepare reactor names and masks before spawning so allocation failures remain recoverable.

Inspired by NVIDIA/cuCascade PRs NVIDIA#240 and NVIDIA#243; retain the existing pool API and cover startup ordering, failure cleanup, and real REST worker affinity.

NVIDIA#240
NVIDIA#243

Amp-Thread-ID: https://ampcode.com/threads/T-01a1c140-9c31-7237-bc6b-617fa475f82a
Co-authored-by: Amp <amp@ampcode.com>
@ivanmilevtues ivanmilevtues changed the title Use shared thread utilities for pool naming and affinity Make pool and I/O reactor startup safe under pinned creators Nov 9, 2026
@ivanmilevtues

Copy link
Copy Markdown
Member Author

/codeboarding

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

CodeBoarding review - not configured

llm: license needs license_key, which is empty.

1. Add a repository secret named CODEBOARDING_LICENSE, with your CodeBoarding licence key as the value.

2. In .github/workflows/codeboarding.yml, the CodeBoarding step's with: block needs to read:

        with:
          llm: license
          license_key: ${{ secrets.CODEBOARDING_LICENSE }}

No analysis ran, and no CodeBoarding hosted usage was consumed.

run 42667198403 - attempt 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant