[Karakatiza666]

Closes #551

Motivation

Http.send! and Http.send_streaming! block until the response headers arrive, and the only way to stop waiting is a timeout, which cancels the request. That makes this pattern impossible: a long "commit" request is sent, and while the server works on it the app queries a progress endpoint once per second and prints the result. This PR adds Http.start!, which sends a request and returns a handle without waiting. The app can then wait on the handle with a time limit (running out of time does not cancel anything), send other requests, or cancel.

Roc API

Http.start! : Request, StreamingOptions => Try(PendingResponse, [InvalidUrl(Url.ParseErr), HttpErr(TransportErr)])

Http.PendingResponse.wait_within! : PendingResponse, Duration => Try([Ready(StreamingResponse), NotReady], [Closed, Interrupted, HttpErr(TransportErr)])
Http.PendingResponse.wait!        : PendingResponse => Try(StreamingResponse, [Closed, Interrupted, HttpErr(TransportErr)])
Http.PendingResponse.close!       : PendingResponse => {}

Usage (see examples/http-pending.roc):

commit = Http.start!(commit_request, { idle_timeout: NoTimeout })?
while True {
    match commit.wait_within!(Duration.millis(500))? {
        Ready(response) => {
            body = response.read_to_end!(NoLimit)?
            break
        }
        NotReady => {
            progress = Http.get_utf8!(progress_url, {})?
            Stdout.line!("Progress: ${progress}")?
        }
    }
}

Semantics

• NotReady does not cancel the request. It stays in flight and can be waited on again.
• Err(Interrupted) (a caught signal) keeps the request too. Errors are otherwise the same as send_streaming!, and an expired timeout is Err(HttpErr(Timeout)) on every later wait.
• Releasing the last reference to the PendingResponse, or close!, cancels the request and closes its connection. A response already returned stays readable. After close!, waits return Err(Closed).
• Redirects follow the same rules as send_streaming!: both use the shared redirect_step!. One deadline covers all hops. wait_within!(Duration.zero) that sees a redirect only starts the next hop, so it returns NotReady until that hop's response has arrived.
• Both timeouts count from start!, not from the wait. The request timeout is the deadline for the whole exchange; idle_timeout bounds each hop's wait for status and headers. A head that arrived in time (judged by arrival time recorded by the task) is returned by a later wait, however late. A slow server needs a larger idle_timeout, or NoTimeout, as in the example.
• start! fails only for an invalid URL or when the host cannot allocate a handle. Failures of the request itself come from the waits.

Progress model

Option A, as in the issue: there is no background thread. A pending request is sent and answered only while the app is inside an HTTP call or a wait (start!, send!, send_streaming!, a body read, wait_within!, wait!). In between, the kernel buffers what arrives. A large request body, or a server that sends a lot before its status line, can stall until the next such call. wait_within! also drives all other pending requests, so a loop of waits needs no Sleep. This is documented on Http.start!.

Option B, a background thread driving the runtime, is a possible follow-up. It is not in this PR because it changes the runtime and shutdown rules documented in src/http.rs, and risks the exit hang described there. The API above would not need to change for it, so it can follow separately.

Host design

• New resource kind HttpPending (platform/Resource.roc, src/resources.rs). http_start! spawns a tokio task on TOKIO_RUNTIME that sends the request and notes when the head arrives.
• Waits go through the existing await_io, so signals and the other waits behave as before.
• Four hosted functions in platform/Host.roc: http_start!, http_pending_wait!, http_pending_follow!, http_pending_close!.
• Redirect state lives in the host handle. http_pending_follow! replaces the request behind the handle with the next hop and records its (status, Location); each Ready payload of http_pending_wait! carries the list of followed redirects, and Roc replays it (current_hop) to recover the current request, URL and remaining count.
  • Why: a PendingResponse is an immutable Roc value, so it cannot carry the current hop across a NotReady; the hop has to be remembered somewhere that outlives one call.
  • Rejected alternatives:
    • Returning an updated value as NotReady(PendingResponse): older copies go stale, and every caller has to thread the new value through its loop.
    • Following redirects in Rust: this duplicates the security checks and Url handling in redirect_step!, and the two paths could drift apart.
    • Having the host keep the Roc Request and convert it back: this needs a lossy inverse of the request conversion and Roc-owned memory inside a resource.

Choices worth a look

• Every wait returns the same StreamingResponse (as Child.wait! does), so the pending value keeps the body's connection alive while it exists. Copies share one body cursor.
• PendingResponse holds the original Request, body included, because 307/308 replay it.
• close! returns {}, unlike Child.close!, since there is nothing to report: closing twice or after completion is a no-op.
• The method is wait_within!, not poll!: Child.poll! takes no duration and never waits, which is not what this does.

Commits

1. Extract the shared parts of streaming sends: no behaviour change. Moves the streaming client, head conversion, timeout arguments and the redirect decision (redirect_step!) into helpers.
2. Add a host resource for HTTP requests in flight (Rust and hosted functions).
3. Add Http.start! (Roc API, docs, README).
4. Tests, test-server route and example.

Reviewing per commit is easiest; commit 1 can be skimmed for "same behaviour".

Testing

• Rust tests in src/http.rs: not ready then ready, zero wait, other HTTP calls during a request, interruption, drop and close ending the connection, a returned response outliving its pending request, request timeout counting from start, a head that arrived in time outliving the timeouts, idle timeout, transport errors, one deadline across a redirect hop, and the same bytes on the wire as send_streaming!.
• The test server in ci/rust_http_server gets a /delay/<ms> route.
• tests/http-in-flight.roc runs the Roc API against it (named so its binary does not clash with the example's). examples/http-pending.roc shows the progress pattern. Both are in scripts/test_spec.json.
• Each test was checked to fail when the code it covers is reverted.
• Checks run: cargo fmt, clippy, cargo test three times, glue check, roc test, the full scripts/test.py, and an x64musl build with a valgrind run.

Not tested:

• A signal arriving in the middle of a wait. Only the pre-wait check is deterministic and covered.
• InsecureRedirect through a pending request: there is no HTTPS test server, and the check is in the shared redirect_step!.
• The yield turns taken when give_up_at has passed are covered only probabilistically.

Coordination

A separate PR on TLS trust options touches src/http.rs and platform/Http.roc. This PR does not change https_connector() or TLS setup. Small conflicts are possible in streaming_client() and the timeout lines of hosted_http_send_streaming. If that PR adds per-request options to StreamingOptions, Http.start! and http_start! must pass them as well.

🤖 Generated with Claude Code