Skip to content

[Brook B1] Added the streams bridge feature that links the temporalio-streams crate. - #370

Closed
moetemp wants to merge 5 commits into
moe/brook/review/g3-workflow-taskfrom
moe/brook/review/b1-streams-feature
Closed

moetemp wants to merge 5 commits into
moe/brook/review/g3-workflow-taskfrom
moe/brook/review/b1-streams-feature

Conversation

@moetemp

@moetemp moetemp commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

What was changed

This PR adds the streams cargo feature to the bridge, which links the Stream runtime crates and builds a runtime for the Redis and memory backends. temporalio-streams has the runtime and the memory Store, and temporalio-streams-redis has the Redis Store. It also generates the Stream protos that the Core bump brings.

  • The feature is off by default and on in poe build-develop and every wheel build. fips turns it on too.
  • StreamCallError is in every build, so Python can import it with the feature off.
  • temporalio-streams comes from the Core submodule. [patch.crates-io] points the Redis crate's temporalio-common and temporalio-streams at the submodule, so the bridge sees one copy of each type.
  • The TLS and fips features forward to the Redis crate, which links the Redis client.
  • The linters skip the submodule, since it has the crate's Python script of the shared vectors.
  • The Core submodule has the Stream protos, so scripts/gen_protos.py generates them as it does the other Core protos. The three files of feat(brook): add the Stream service messages (as experimental) temporalio/api#895 at 1f72662 go to temporalio.api.stream.v1, temporalio.api.streamservice.v1 and the Stream enums of temporalio.api.enums.v1. The runtime's own messages (coresdk.streams, such as the writer calls and the memory runtime's config) go to temporalio.bridge.proto.streams, which the API docs keep private. No hand-written code uses them yet.
  • Fork-only pins. The submodule is at 872c90fc on moetemp/sdk-rust, the Stream crate of feat: add the Stream protos and WIT to the vendored API temporalio/sdk-rust#1733 and its writers on top of the merged feat(brook): add the dispatching Workflow Task to each Local Activity start temporalio/sdk-rust#1703. Its wire has the StreamRef, a oneof Producer epoch, a strict create with a request id, a StreamWriter for the Producer and withdrawal fields of a write, and a close that writes no Record. Cargo.toml takes temporalio-streams-redis by git from temporal-community/sdk-streams at dc93cba6, and Cargo fetches it with the feature off too. Upstream pins the merged Core commit, takes the Redis crate from crates.io by version, keeps the patch and drops the git source.

Why?

The Stream runtime is Rust, so every SDK shares the Store logic and its rules. A Python build links it like the rest of Core.

Open question. With the feature off by default, a source build from the sdist has no Stream runtime, but the wheels have it. Should it be on by default once the crate is on crates.io?

Part of the Streams review series.

Checklist

  1. How was this tested:
    • poe lint, cargo clippy and cargo fmt on both feature sets, and pytest tests/bridge.
    • scripts/gen_protos.py on the protobuf 3 toolchain leaves no diff.

@moetemp moetemp added the skip-changelog Changelog entry rides another PR label Oct 25, 2026
@moetemp
moetemp force-pushed the moe/brook/review/b1-streams-feature branch 3 times, most recently from 8aa8d7f to 4dd874f Compare October 27, 2026 09:07
@moetemp
moetemp changed the base branch from moe/brook/review/base to moe/brook/review/g3-workflow-task October 27, 2026 09:07
The feature builds the crate's Stream runtime for a backend, on Redis or
on the memory Store. The official wheels and the FIPS build turn it on.
`StreamCallError` is in every build, so Python can map its errors.
@moetemp
moetemp force-pushed the moe/brook/review/b1-streams-feature branch from 4dd874f to f2244db Compare October 28, 2026 01:33
@moetemp

moetemp commented Nov 10, 2026

Copy link
Copy Markdown
Owner Author

Closed in favor of #381, the combined branch, which has this change.

@moetemp moetemp closed this Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Changelog entry rides another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant