Skip to content

Add TLS client connections over Tcp and trust options for Tcp and Http - #552

Draft
Karakatiza666 wants to merge 1 commit into
roc-lang:mainfrom
Karakatiza666:tls-client-streams
Draft

Karakatiza666 wants to merge 1 commit into
roc-lang:mainfrom
Karakatiza666:tls-client-streams

Conversation

@Karakatiza666

Copy link
Copy Markdown

Summary

A basic-cli app cannot open a TLS connection over Tcp, and HTTPS only trusts the bundled WebPKI roots. Consider a CLI client for a self-hosted service that uses HTTPS and wss://. It cannot run a WebSocket client over TLS, and it cannot trust the service's private CA. During development it also cannot accept the service's self-signed certificate. Other TLS protocols over Tcp hit the same limit: Redis, Postgres and SMTP over TLS.

This PR adds:

  • Tcp.connect_tls!, which opens a TLS client connection and returns the existing Tcp.Connection.
  • A Tls module with trust options shared by Tcp and Http:
    • Default: the bundled WebPKI roots, unchanged.
    • AddPem(bytes): the bundled roots plus certificates from PEM, like curl's --cacert.
    • DangerAcceptInvalidCerts: accept any certificate, like curl's --insecure.
  • A tls field on Http.SendOptions and Http.StreamingOptions.
  • TlsErr(Tls.Err): a handshake failure names its cause: UnknownIssuer, Expired, NameMismatch, BadPem(Str) or Other(Str).

Related to #454. This PR does not change the default trust source (OS store vs. bundled roots), which #454 is still researching. It only adds explicit, app-chosen trust on top of the current default, and any outcome of #454 would change only what Default means. It also covers the TLS part of #438 ("map TLS failures intentionally"). #331 was an earlier attempt at TLS over Tcp; it was closed without review.

Design

Roc API

Tls :: [].{
	Trust : [Default, AddPem(List(U8)), DangerAcceptInvalidCerts]
	Options := { trust : Trust ?? Default }
	Err := [UnknownIssuer, Expired, NameMismatch, BadPem(Str), Other(Str)].{ is_eq : _, to_str : Err -> Str }
}

# Tcp
ConnectTlsOptions := { connect_timeout : Duration, io_timeout : Duration, tls : Tls.Options ?? {} }
connect_tls! : Str, U16, ConnectTlsOptions => Try(Connection, [Interrupted, TlsErr(Tls.Err), TcpErr(IOErr)])

# Http
SendOptions := { max_redirects : U64 ?? 0, tls : Tls.Options ?? {} }
StreamingOptions := { idle_timeout : …, max_redirects : U64 ?? 0, tls : Tls.Options ?? {} }
TransportErr : [ …, TlsErr(Tls.Err)]
ca = Path.read_bytes!("ca.pem")?
conn = Tcp.connect_tls!("db.internal", 6380, { connect_timeout: Duration.seconds(5), io_timeout: Duration.seconds(30), tls: { trust: AddPem(ca) } })?

Why this shape:

  • connect_tls! returns the existing Tcp.Connection. All of its operations work unchanged over TLS: reads, writes, with_timeout, line limits, chunks and lines. So does every package already written against Tcp.Connection, such as a pure-Roc WebSocket client, which gets wss:// for free.
  • It is a separate function, not a tls field on connect!. A TLS connect has a different failure set. Adding TlsErr to connect!'s error row would break every exhaustive match on it.
  • One Tls module serves both Tcp and Http. It holds one vocabulary for trust and for errors. Http does not import Tcp just to name a type, and future TLS settings (client certificates, ALPN) have an obvious home. The module follows the nested nominal style of Sqlite.Err / Sqlite.OpenOptions, and Err has is_eq and to_str as CONTRIBUTING asks.
  • The Http option goes on SendOptions / StreamingOptions. It does not go on the request, because Request (and so timeout) belongs to the external http package. These two records already carry the platform-owned per-call transport policy (max_redirects), and they are what reaches send_redirected!. The trust is passed to the host on every hop, so redirects use it too. That includes cross-origin hops: the next host is verified against the same trust.
  • The insecure mode is hard to pick by accident. It is only reachable by writing DangerAcceptInvalidCerts, and it is never a default. Even then, the handshake signatures are still verified with ring, so the server must hold the key for the certificate it presents.
  • Every field is optional, with a default. {} keeps today's behaviour. BadPem is reported before any DNS lookup, connection or I/O, even for http:// URLs, so a broken CA file is never hidden.
  • TlsErr sits inside HttpErr(...) and is not a new outer tag. The result rows of send!, get!, send_streaming! and the others are unchanged.

Host

  • src/tls.rs (new) holds the TLS code:
    • It builds a rustls::ClientConfig per trust. The default and insecure configs are built once. PEM configs are cached in a small cache keyed by the PEM bytes.
    • It has the insecure verifier and maps rustls errors to Tls.Err, including the …Context variants.
    • It finds a rustls error inside hyper's nested errors.
    • It drives rustls on non-blocking sockets.
  • The Tcp changes stay inside the existing connection type.
    • BufferedSocket gets an optional rustls::ClientConnection, so there is no new resource kind.
    • Data rustls has already decrypted is read before any wait, so a read never blocks while data is available.
    • The deadline, the 10 ms signal poll and the rule that a timed-out read consumes nothing are unchanged.
    • When the session has output pending, the wait wakes for either direction, so a peer that is blocked writing to us cannot stall a read.
    • Dropping a connection sends close_notify without waiting.
    • SNI and verification use the host string the app passes. An IP literal is checked against the certificate's IP addresses.
  • The src/http.rs change is small.
    • https_connector takes the config (with_tls_config, with the same https_or_http / http1 features).
    • hosted_http_send_streaming takes the trust as one new argument.
    • A handshake failure becomes Failure::Tls.
    • The send, idle-timeout, body and redirect logic is untouched.
  • Dependencies: no new crates, and no C dependency. rustls moves from dev-dependencies to dependencies, and webpki-roots becomes a direct dependency. Both were already linked through hyper-rustls. THIRD_PARTY_LICENSES.md is unchanged.

Compatibility

  • Existing apps compile and behave the same, with one exception: code that matches Http.TransportErr exhaustively, without a _ arm, needs a TlsErr arm. The module has added variants before (RedirectLimitExceeded … InsecureRedirect), and no example or test matches it exhaustively.
  • HTTPS connections that use the default trust now share one ClientConfig, so TLS session resumption can happen across requests. Configs are per trust, so a session never resumes across trusts.

Questions for review

  1. A peer that closes without close_notify is reported as TcpErr(Other(…)) rather than end of input, because data may have been truncated. This matches rustls. Some servers close TCP without close_notify. Is strict the right default, or should it be a distinct, matchable signal?
  2. Should there be OnlyPem(bytes), which replaces the bundled roots instead of adding to them? It is easy to add later.
  3. DangerAcceptInvalidCerts carries across cross-origin redirects, as curl's -k does. Is that the behaviour we want?
  4. There is no server-name override. Connecting by IP to a certificate that only has a DNS name gives NameMismatch.

Verification

Tests:

  • Rust tests: 24 new ones, against a local rustls server that uses the existing tests/tls fixtures. The fixtures add PEM forms and an unrelated key. No rcgen is needed: "expired" uses a fixed clock in 2040.
    • Trusted by AddPem, rejected by Default (UnknownIssuer).
    • NameMismatch for 127.0.0.1, Expired, and accepted by DangerAcceptInvalidCerts.
    • DangerAcceptInvalidCerts still rejects a server whose key does not match its certificate.
    • BadPem is reported before connecting.
    • The handshake respects the connect deadline, fails when the server closes during it, and reports Interrupted.
    • A timed-out TLS read loses no bytes. Already-decrypted data is read without waiting. An interrupted read keeps its prefix, and line limits apply.
    • Clean and unclean close_notify; dropping a connection sends close_notify.
    • The either-direction wait.
    • Error conversion to Roc, and HTTPS through send_streaming_at and the hosted function.
  • Revert checks: every new test was checked by temporarily reverting the code it covers, and each one failed. Three of them (TLS timed-out reads, interrupted reads and line limits) can only be broken by reverting machinery shared with plain TCP. They guard against TLS bypassing that machinery rather than isolating TLS code.
  • examples/tcp-tls.roc and tests/tls-trust.roc run against a new stdlib-only helper, scripts/tls_test_server.py: TLS echo on 8443 and HTTPS on 9443, with no internet access.
    • Tcp: the trust matrix, BadPem against a closed port (no I/O first), and reads and lines over TLS.
    • Http: the trust matrix, same-origin and cross-origin redirects, and streaming.

Commands, all passing:

  • cargo fmt --all -- --check
  • cargo clippy --locked --all-targets -- -D warnings. It also passes for --lib --tests on x86_64-pc-windows-gnu using zig cc, to type-check the Windows poll path.
  • cargo test --locked --all-targets: 224 passed on Linux.
  • ./ci/regenerate_glue.sh --check
  • roc check and roc test platform/main.roc: 764 tests passed.
  • python3 scripts/third_party_notices.py --check
  • ROC=/path/to/nightly-2026-11-06-34f137a/roc python3 scripts/test.py: the full native Linux suite passed.
  • scripts/build.py --target arm64musl built.
  • macOS and Windows native runs are left to CI.

🤖 Generated with Claude Code

Tcp.connect_tls! opens a TLS client connection and returns the existing
Tcp.Connection, so reads, writes, timeouts, line limits, chunks and lines
work as for Tcp.connect!. The handshake runs under connect_timeout and is
interruptible; decrypted data is taken before any wait, so a timed-out read
still loses nothing.

The new Tls module holds the shared settings. Tls.Options chooses which
server certificates are accepted: the bundled WebPKI roots (the default,
unchanged), the bundled roots plus certificates from PEM, or
DangerAcceptInvalidCerts. Http.SendOptions and Http.StreamingOptions take
the same tls field, applied on every redirect hop. A failed handshake is
TlsErr(Tls.Err), naming an unknown issuer, an expired certificate, a name
mismatch, unusable PEM (reported before any network activity) or another
cause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant