Skip to content

Unify URMA Jetty resource creation and ownership - #102

Open
Spicy-cream wants to merge 1 commit into
masterfrom
urma-resource-ownership
Open

Spicy-cream wants to merge 1 commit into
masterfrom
urma-resource-ownership

Conversation

@Spicy-cream

@Spicy-cream Spicy-cream commented Nov 17, 2026 •

Copy link
Copy Markdown

What problem does this PR solve?

URMA startup preallocation and on-demand connection setup each construct the same JFCE, JFC, JFR, and Jetty. The paths also interpret queue-depth flags independently, so a prepared resource can be selected using current flags instead of the configuration it was created with.

What changed?

  • Use one factory for local queues in both paths, with cleanup on partial creation failure.
  • Give the endpoint and the prepared cache exclusive ownership of local queues. Keep remote Jetty and segment imports scoped to the connection.
  • Normalize queue depths in both paths and match prepared resources against their stored creation configuration.

This is the first change in the Jetty resource pooling series. Used Jetty resources are still destroyed when their connection closes.

Validation

  • Built the URMA mock configuration on Linux and ran brpc_urma_unittest.
  • Added mock coverage for queue configuration, event and polling configurations, and JFCE descriptor cleanup. Hardware validation is reserved for the resource recycling change.

Follow-up PRs for Jetty resource pooling

  1. Bound the pool of prepared resources. Add configuration-matched checkout, capacity accounting, and background replenishment with backoff. Verify that simultaneous checkouts preserve exclusive ownership and respect the configured resource budget. This stage still destroys resources after use.
  2. Retire resources safely. Stop new work requests and synchronize teardown with both CQ polling and event callbacks. Keep buffers alive until the provider confirms that DMA has stopped. Exercise teardown races, posted but unused receives, and late completions.
  3. Recycle resources on supported providers. Once steps 2 and 3 are in place, reset the Jetty and its JFR, drain completions and notifications, and remove the old peer association before returning resources to the pool. Enable reuse only for provider configurations verified on real hardware to prevent old traffic from reaching a new connection. Start with reuse disabled by default and destroy resources when reuse checks fail.
  4. Add idle trimming, diagnostics, and operational validation. Trim idle resources, retire resources from invalidated device contexts, and expose pool usage and retirement failures. Compare on-demand allocation, preallocation, and recycling under repeated connects and disconnects. Report connection setup p99, resource creation counts, CPU, and retained resources.

Steps 2 and 3 can proceed independently after this PR. Each follow-up will preserve existing unit tests and add focused cases as needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant