feat(cli): migrate a Postgres relay into LMDB, with resume, verify and a reported throughput

.changeset/pg-to-lmdb-migration.md

@@ -0,0 +1,2 @@
+---
+---

.github/workflows/checks.yml

@@ -273,12 +273,12 @@ jobs:
           -D clippy::todo -D clippy::unimplemented -D clippy::panic
       - name: Tests
         run: cargo test --workspace --all-features
-      # The two database-backed storage targets create and migrate their own scratch
-      # databases; the Cucumber half of the harness stays in scripts/storage-parity.sh.
+      # The database-backed storage targets create and migrate their own scratch databases; the
+      # Cucumber half of the harness stays in scripts/storage-parity.sh.
       - name: Storage parity against Postgres
         env:
           PARITY_ADMIN_URL: postgres://postgres:postgres@localhost:5432/postgres
-        run: cargo test -p nostream-storage --test migrations --test parity -- --ignored --test-threads=1
+        run: cargo test -p nostream-storage --test migrations --test parity --test migration_pg -- --ignored --test-threads=1
       - name: Storage builds without the Postgres driver
         run: cargo check -p nostream-storage --no-default-features --all-targets
       - name: Release build

.gitignore

@@ -55,3 +55,7 @@ coverage-report/
 
 # Rust
 /target/
+
+# A migration target is the dataset: `--target .` writes it here, and this repository is public.
+/lmdb/
+/migration-manifest.json

CLAUDE.md

@@ -36,17 +36,19 @@ cargo test --workspace --all-features
 cargo build --workspace --all-features --release
 ```
 
-CI additionally runs the two database-backed storage tests ("Storage parity against Postgres")
+CI additionally runs the three database-backed storage tests ("Storage parity against Postgres")
 against a `postgres:16-alpine` service, and they are `#[ignore]`d so a checkout without a database still passes
 `cargo test --workspace`:
 
 ```
-cargo test -p nostream-storage --test migrations --test parity -- --ignored --test-threads=1
+cargo test -p nostream-storage --test migrations --test parity --test migration_pg \
+  -- --ignored --test-threads=1
 ```
 
 `PARITY_ADMIN_URL` names the administrative database (CI sets
 `postgres://postgres:postgres@localhost:5432/postgres`); the targets create, migrate and
-drop their own scratch databases. `scripts/storage-parity.sh` runs both plus the
+drop their own scratch databases — `migration_pg` seeds one and copies it into a temporary LMDB
+store. `scripts/storage-parity.sh` runs all three plus the
 TypeScript Cucumber suite against the sqlx-migrated schema — the complete local oracle. The
 LMDB arm of that oracle needs no database: `cargo test -p nostream-storage --test parity_lmdb
 -- --nocapture` replays the same fixture on a temporary store and runs inside the fifth
@@ -88,7 +90,7 @@ resolver 3:
 | `nostream-relay` | connections, wire protocol, handlers, rate limiters, live dispatch, NIP-77 engine (the acceptor lives in `nostreamd`) |
 | `nostream-services` | payment processors and the invoice lifecycle, operator notifications, the web-of-trust graph, Tor, the scheduler and its workers, event import, NIP-66 monitors (#865) |
 | `nostream-admin` | admin API and auth |
-| `nostream-cli` | operator command line (`nostream-cli config validate`, `nostream-cli import`) |
+| `nostream-cli` | operator command line (`nostream-cli config validate`, `nostream-cli import`, `nostream-cli migrate pg-to-lmdb`) |
 | `nostreamd` | the daemon binary: runtime, telemetry, signals, the services composition root and the worker supervisor |
 
 - Every crate manifest repeats `version.workspace = true`, `edition.workspace = true`,

crates/nostream-cli/Cargo.toml

@@ -14,16 +14,21 @@ path = "src/main.rs"
 [dependencies]
 clap.workspace = true
 anyhow = { workspace = true }
+hex.workspace = true
 nostream-config.workspace = true
 nostream-core.workspace = true
 nostream-services.workspace = true
 nostream-storage.workspace = true
+serde.workspace = true
 serde_json.workspace = true
 serde_yaml.workspace = true
+sha2.workspace = true
 thiserror.workspace = true
 tokio.workspace = true
 
 [dev-dependencies]
+chrono.workspace = true
+heed.workspace = true
 tempfile.workspace = true
 
 [lints]