Introduction

KesselDB is a deterministic, replicated SQL database. Speaks PostgreSQL, HTTP, WebSocket, and a fast binary wire. Zero-dependency Rust kernel.

"It's the database that made the Kessel Run in 12 parsecs."

This is the official documentation site, built from the canonical README.md, docs/USAGE.md, docs/ARCHITECTURE.md, docs/STATUS.md, and AGENTS.md files in the repository. The book and the in-repo markdown are a single source of truth — when one updates the other ships the same change on the next push to main.

The README is the front door, with the pitch + capabilities + quick start. Read it first:

What is KesselDB?

KesselDB is a from‑scratch Rust database with the engineering rigor of TigerBeetle — deterministic state machine, LSM storage, write‑ahead log, Viewstamped Replication, simulation‑driven testing — applied to a general, schema‑flexible SQL database instead of a single hard‑coded domain.

You get runtime‑defined tables and online DDL, real SQL (joins, aggregates, indexes, constraints, triggers, transactions), exactly‑once client semantics across a replicated multi‑node cluster, and four wire protocols on the same engine:

• Binary — the deterministic fast path (Op::encode() length‑prefixed frames)
• HTTP/1.1 + JSON — /v1/sql, /v1/op, /v1/health, /v1/metrics (Prometheus)
• WebSocket — long‑lived /v1/ws upgrade, framed Op::encode() payloads
• PostgreSQL Frontend/Backend v3.0 — psql, pgcli, JDBC, psycopg, pgx, tokio-postgres, sqlx-pg + GUI tools (pgAdmin, DBeaver, DataGrip, Metabase, Tableau) connect straight in

The kernel is pure Rust with zero external dependencies. Every wire surface is opt‑in via cargo features — cargo build --release links no gateway code at all. Determinism is a feature, not an aspiration.

Highlights

• Real SQL — CREATE TABLE, ALTER TABLE … ADD COLUMN (online, no lock), DROP TABLE, INSERT, SELECT (filters incl. IN / BETWEEN / LIKE / IS [NOT] NULL / AND/OR/NOT, JOIN (INNER/LEFT/RIGHT/FULL on a binary join, chained 3+ table INNER, table aliases users u / users AS u), GROUP BY, HAVING, ORDER BY, LIMIT/OFFSET), UPDATE, DELETE, COUNT/SUM/MIN/MAX/AVG, CREATE [UNIQUE|RANGE] INDEX, DESCRIBE, EXPLAIN.
• Constraints & logic — NOT NULL, UNIQUE, foreign keys ENFORCED from CREATE TABLE … FOREIGN KEY DDL (bad child INSERT → SQLSTATE 23503, NULL FK allowed) with ON DELETE NO ACTION/RESTRICT/CASCADE/SET NULL/SET DEFAULT, CHECK, and deterministic triggers (a gas‑bounded zero‑dep expression VM) — incl. zero‑dep, test‑vector‑verified SHA‑256 / HMAC‑SHA256 usable in CHECK/triggers (pgcrypto‑subset).
• Atomic transactions — SQL BEGIN/COMMIT/ROLLBACK (and op‑level Op::Txn): all‑or‑nothing, replicated as a single operation. Multi‑row INSERT … VALUES (…),(…) is one atomic op in one round‑trip — a naive client pays N round‑trips and N consensus decisions; KesselDB pays one.
• Replicated & highly available — Viewstamped Replication over real TCP sockets; safety‑hardened (no committed‑op loss across view change) and liveness‑tested under an adversarial partition corpus.
• Exactly‑once clients with automatic failover — stable client sessions; a ClusterClient finds the primary, retries safely, and never double‑applies.
• Crash‑safe — WAL replay with torn‑tail handling; tested.
• Operable — hot consistent snapshots/backup, live metrics, shared‑secret auth, connection quotas and backpressure.
• Fast where it counts — prepared‑statement cache (≈26× faster SQL compile), per‑SSTable bloom filters, bounded‑segment compaction for data‑size‑independent point reads, range/band index narrowing, a columnar fast‑path that answers MIN/MAX from the index extreme without scanning, and an in‑memory read cache for hot keys — all on by default, each proven equivalent to a full scan by a randomized oracle.
• Mechanically verified by TLA+ (S1) — the Viewstamped Replication safety invariants are model‑checked by TLC across 528 million distinct states / depth 21 (zero counterexamples). Seven layered TLA+ modules cover the full Replication → MVCC backbone (Replication / MVCCStorage / MVCCTx / MVCCSi / MVCCSsi / MVCCGc / MVCCCutover). See kesseldb-tla/.
• Serializable MVCC (S2) — every SQL statement that touches a user-type row is, by construction, a deterministic MVCC transaction with snapshot-isolation
  • Cahill SSI (write‑skew impossible) + GC under a dynamic watermark protocol. Replicas reach byte‑identical state at every committed log position.
• Jepsen-style linearizability under partition (S3) — 5 hand-derived Jepsen tests against the in-process VSR + MVCC stack; multi-replica byte-identity digests post-partition + post-recovery.
• Deterministic WASM UDFs (S4) — zero‑dep WASM-MVP interpreter (kessel-wasm) for CHECK/trigger user functions: i32/i64/f32/f64 + memory + tables/call_indirect + canonical NaN, gas‑bounded, no host calls / no clocks — every replica runs byte‑identical UDF logic; UDF behavior is replayable from the log.
• External sources & Parquet — register and REFRESH JSON/NDJSON/CSV/Parquet from HTTP/HTTPS endpoints or directly from S3‑compatible and Azure Blob object storage. The pure‑Rust zero‑dep Parquet reader (kessel-parquet) supports **flat REQUIRED + OPTIONAL + LIST<primitive> + MAP<K, V> + struct (+ 3‑deep cross‑products) × UNCOMPRESSED + Snappy + GZIP + zstd
  • LZ4_RAW + Brotli (the full 6‑codec matrix) × PLAIN + dictionary × V1 + V2 data pages × INT64 + INT32 + INT96 (timestamps) + DECIMAL (INT32 / INT64 / FLBA, precision ≤ 38) + FLBA + BYTE_ARRAY** out of the box. Every nested Parquet shape pyarrow writes up to 3‑deep nesting decodes. See Parquet capability matrix below. (--features external-sources, default off; --features external-sources-objstore for S3/Azure + Parquet; deterministic kernel unaffected when off.)
• Cross‑shard scatter scan — SELECT / SELECT … ORDER BY / projection / row‑filter ops fan out across K independent VSR shard groups via a zero‑dep std‑thread scatter‑gather with bounded per‑shard channels. Unordered scan is shard‑id deterministic; sorted scan is a BinaryHeap k‑way merge of per‑shard already‑sorted streams. K‑invariance locked by an 85‑seed × 5‑K property sweep: with unique sort values, merged output is byte‑identical to the K=1 baseline for K ∈ {1, 2, 4, 8, 16}. Per‑shard MVCC snapshot per request; opt‑in best‑effort partial_on_timeout mode beside the safe hard‑fail default.
• HTTP/1.1 gateway (opt‑in --features http-gateway) — full Op surface
  • SQL + /v1/health + /v1/metrics (Prometheus text v0.0.4) on a sibling TCP listener (ServerConfig.http_addr; HTTPS on http_tls_addr with the tls feature). Authorization: Bearer constant‑time, optional X-Kessel-Client-Id + X-Kessel-Req-Seq exactly‑once headers. JSON responses via the existing kessel_client::format_result_json contract. Binary protocol byte‑untouched; zero external (non‑workspace) deps on the gateway crate. See docs/USAGE.md §HTTP gateway.
• WebSocket gateway (shipped under the HTTP gateway crate) — long‑lived /v1/ws upgrade carrying raw Op::encode() payloads under the kessel-op-v1 subprotocol. RFC 6455 strict handshake, binary frames only, bounded send queue (16 messages), 30 s ping/pong heartbeat. Same Bearer auth as HTTP, checked once at handshake. Useful for browser‑direct push/streaming clients that don't want a per‑request HTTP round trip. Enabled automatically with --features http-gateway. See docs/USAGE.md §HTTP gateway → WebSocket.
• PostgreSQL wire protocol (opt‑in --features pg-gateway) — Frontend/Backend Protocol v3.0 Simple Query AND Extended Query paths with SCRAM‑SHA‑256 authentication on a sibling TCP listener (ServerConfig.pg_addr, default port 5432). Operator's Bearer token IS the SCRAM password input — one credential surface; rotating the token rotates HTTP, WS and PG together. SELECT / INSERT / UPDATE / DELETE / CREATE TABLE work end‑to‑end against psql, pgcli, JDBC, psycopg, pgx, tokio-postgres, sqlx-pg, Diesel‑pg, GORM‑pg, Drizzle‑pg, Prisma‑pg, and every libpq‑derived client. The full Extended Query message set (Parse / Bind / Describe / Execute / Sync / Close / Flush) ships — psycopg2's cursor.execute("…WHERE id = %s", (42,)) round‑trips end‑to‑end, and ORMs that REQUIRE prepared statements (SQLAlchemy, Drizzle, Prisma, JDBC default) connect. pg_catalog + information_schema stubs let pgAdmin 4, DBeaver, DataGrip, Metabase, Tableau, Looker, dbt and pgJDBC getTables all connect + browse out of the box. Cap‑overflow (53300) and idle‑timeout (57014) emit wire‑level ErrorResponse with canonical PG message text before closing. Independent connection cap from HTTP (default pg_max_conns=256 vs HTTP's 1024) — a misbehaving pgcli cannot starve HTTP clients. Binary protocol byte‑untouched; zero external (non‑workspace) deps on the gateway crate. See docs/USAGE.md §9 PostgreSQL clients.
• Deterministic & verifiable — the whole engine is a seedable state machine; the test suite (~2,700+ tests by default, more with the PostgreSQL and HTTP gateway features) includes seeded partition/fault simulation, multi‑replica Jepsen, hand‑derived KATs against published spec text for every codec, the 85‑seed cross‑shard K‑invariance sweep, a synthetic‑peer suite verifying each GUI tool's verbatim introspection SQL, and adversarial pentests for every public input surface.

Quick start

Download a prebuilt Linux binary

# x86_64 Linux (glibc):
VER=v2.0.0       # see https://github.com/hassard0/KesselDB/releases for the latest
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/kesseldb-$VER-x86_64-unknown-linux-gnu \
  -o kesseldb && chmod +x kesseldb
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/kessel-$VER-x86_64-unknown-linux-gnu \
  -o kessel    && chmod +x kessel

# Or grab the bundle (server + CLI + README + USAGE + LICENSE):
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/kesseldb-$VER-x86_64-unknown-linux-gnu.tar.gz \
  | tar xz

The release workflow builds these as part of cargo build --release --features pg-gateway,http-gateway so the binaries you download include the PostgreSQL and HTTP gateways out of the box; the binary protocol is the default + fast path either way.

Or run the official Docker image

A pre-published multi-arch (linux/amd64 + linux/arm64) image is pushed to GitHub Container Registry on every v* release. The image bundles the same --features pg-gateway,http-gateway server you would build from source, runs as a non-root UID, and exposes all three wire surfaces (binary 6532, HTTP+WS 6533, PostgreSQL 5432).

docker run --rm \
  -p 6532:6532 -p 6533:6533 -p 5432:5432 \
  -v $PWD/kesseldb-data:/data \
  -e KESSELDB_TOKEN=changeme \
  ghcr.io/hassard0/kesseldb:latest

Stripped image size: ~77 MiB. See Dockerfile and docs/USAGE.md for the layout + the matrix of supported env vars.

Or build from source

git clone https://github.com/hassard0/KesselDB && cd KesselDB
cargo build --release                                # default — binary protocol only
cargo build --release --features pg-gateway,http-gateway   # all wire surfaces
cargo test  --workspace --release                    # workspace gate: ~2,700+ default tests

Start a node

# kesseldb [LISTEN_ADDR] [DATA_DIR]
./kesseldb 127.0.0.1:7878 ./data

# Or enable the HTTP + PG listeners alongside the binary protocol:
KESSELDB_TOKEN=mysecret \
KESSELDB_HTTP_ADDR=127.0.0.1:8080 \
KESSELDB_PG_ADDR=127.0.0.1:5432 \
  ./kesseldb 127.0.0.1:7878 ./data
# => KesselDB listening on 127.0.0.1:7878, data dir ./data, http=127.0.0.1:8080, pg=127.0.0.1:5432

Connect

# Binary protocol via the kessel CLI (one-shot, pipe, interactive):
./kessel "CREATE TABLE acct (owner U32 NOT NULL, bal I64 NOT NULL)"
./kessel "INSERT INTO acct ID 1 (owner, bal) VALUES (100, 50)"
./kessel "SELECT SUM(bal) FROM acct WHERE owner = 100"     # => = 50
./kessel "SELECT * FROM acct"                              # aligned table
./kessel --json "SELECT * FROM acct"                       # {"status":"ok","rows":[…]}
echo "SELECT * FROM acct ID 1" | ./kessel                  # pipe a .sql file
./kessel                                                   # interactive shell (\? for commands)

# HTTP/1.1 + JSON:
curl -s -X POST --data-binary 'SELECT * FROM acct' \
  -H 'Content-Type: text/plain' \
  -H 'Authorization: Bearer mysecret' \
  http://127.0.0.1:8080/v1/sql

# PostgreSQL wire — any libpq-derived client (Simple Query + Extended Query both supported):
PGPASSWORD=mysecret psql -h 127.0.0.1 -p 5432 -U test "SELECT SUM(bal) FROM acct"
PGPASSWORD=mysecret pgcli -h 127.0.0.1 -p 5432 -u test

# psycopg2 — parameterized queries through the Extended Query protocol:
import os, psycopg2
conn = psycopg2.connect(host="127.0.0.1", port=5432, user="test",
                        password=os.environ["KESSELDB_TOKEN"], dbname="kessel")
cur  = conn.cursor()
cur.execute("SELECT * FROM acct WHERE owner = %s", (100,))
print(cur.fetchall())     # → real rows, real round‑trip

The kessel CLI is one-shot, pipe, and interactive, with reliable exit codes and a --json mode — ideal for scripts, ops, and agents. In the shell, \? lists commands, \d <table> describes a table, \timing toggles query timing.

Or from Rust

#![allow(unused)]
fn main() {
use kessel_client::Client;

let mut db = Client::connect("127.0.0.1:7878")?;

db.sql("CREATE TABLE acct (owner U32 NOT NULL, bal I64 NOT NULL)")?;
db.sql("INSERT INTO acct ID 1 (owner, bal) VALUES (100, 50)")?;
db.sql("INSERT INTO acct ID 2 (owner, bal) VALUES (100, 999)")?;

let total = db.sql("SELECT SUM(bal) FROM acct WHERE owner = 100")?; // => 1049
db.sql("UPDATE acct ID 1 SET bal = 500")?;
let row   = db.sql("SELECT * FROM acct ID 2")?;
}

→ Full instructions, SQL reference, cluster setup, auth and operations are in docs/USAGE.md.

Deploy

Single-pod (default) PLUS replicated VSR cluster mode (--set cluster.enabled=true). Multi-region (cross-zone WAN-tolerant view-change) and sharding × clustering are roadmap follow-ups.

Full walkthrough + caveats (TLS, single‑attach volume, GHCR visibility) in docs/USAGE.md §11; cluster mode + primary-kill failover + Prometheus monitoring in §11.5. The Helm chart (single-pod and cluster mode, including primary-kill failover) is tested end‑to‑end in CI.

PostgreSQL client compatibility

KesselDB speaks the PostgreSQL Frontend/Backend Protocol v3.0 Simple Query AND Extended Query paths with SCRAM‑SHA‑256 auth. With the pg_catalog / information_schema stubs and the Extended Query message set, the following PG ecosystem tools connect and browse out of the box (verified by synthetic‑peer KATs driving each tool's verbatim connect / introspection SQL, and — for the real drivers below — by end‑to‑end driver round‑trips):

This release ships Extended Query (Parse/Bind/Describe/Execute/Sync/Close/Flush) with binary‑format parameters AND binary‑format results for the 10 supported PG scalar types (INT2/INT4/INT8/FLOAT4/FLOAT8/BOOL/ TEXT/VARCHAR/BYTEA/TIMESTAMPTZ); JDBC simple‑mode ::cast rewrite + paren‑VALUES parse + scalar‑SELECT Describe synthesizer; CHAR(N) padding‑aware comparison; COPY FROM/TO STDIN in text, CSV, and binary formats with 181.9× ingest lift.

Follow‑ups on the roadmap: binary NUMERIC, JSONB/UUID/ARRAY binary, pgJDBC simple‑mode nested casts, Describe for multi‑projection SELECTs, Go pgx / Node Drizzle+Prisma smoke harnesses, libpq pipeline mode, RETURNING, CancelRequest action, GUC plumbing, pg_proc real function listing, pg_stat_* runtime stats, TLS via SSLRequest, MD5 auth fallback, SCRAM channel binding, per‑user privileges. Full list in docs/USAGE.md §9 → Limitations.

Running a cluster

A replicated cluster is composed with the kesseldb-server library (spawn_node + serve_clients); clients use ClusterClient, which discovers the primary and fails over automatically with exactly‑once semantics:

#![allow(unused)]
fn main() {
use kessel_client::ClusterClient;

let mut db = ClusterClient::new(vec![
    "10.0.0.1:7878".into(),
    "10.0.0.2:7878".into(),
    "10.0.0.3:7878".into(),
]);
db.call(&op)?; // routed to the primary; retried safely on failover
}

See docs/USAGE.md → Running a cluster.

Performance

Single deterministic writer, default zero‑dependency build. Measured on a 16‑core x86‑64 Linux reference server (numbers move with hardware; the relationships hold across platforms — see docs/PERFORMANCE.md for the scaling model and cloud projections).

Cross‑DB benchmark suite. Full tables — including the losses where KesselDB does NOT win — are in docs/BENCHMARKS.md. Honest summary:

How the headline wins were built:

• Transaction‑bracket read paths. Static all‑RO Op::Txn classification recurses into the inner‑op vector and routes through the read‑pool bypass when every inner op is read‑only — lifting OLTP read‑only at N=16 42.6× (680 → 28,977 tx/s) to 5.7× Postgres, and now 6.02×.
• Mixed‑transaction split‑phase dispatch. (R*, W*)‑shape mixed Txns are split at the read/write boundary — the read prefix runs in parallel via the read‑pool bypass, the write suffix serially via sm.write().apply; read‑after‑write Txns fall through to unified apply (byte‑equivalent preserved via apply's overlay). This lifts OLTP read‑write at N=16 14.4× (712 → 10,273 tx/s) to 2.30× Postgres.
• Sharding. K independent per‑shard sub‑engines (each its own Arc<RwLock<StateMachine>> + apply thread + WAL + SSTables, rooted at data_dir/shard‑<i>/) route every Op via hash(make_key(type_id, oid)) % K. Opt‑in via ServerConfig.shard_count = Some(K) (default None is byte‑identical to the unsharded engine). Get‑by‑id scales K=1 ~4.9M → K=4 ~11.4M (2.3×) → K=8 ~14.7M (3.0×, breaks the 10M ceiling) → K=16 ~16.2M (3.3×); p50 latency drops 3 µs → <1 µs. Scan‑side companions prove K‑invariance for scatter‑gather scans, recover find‑by perf at K≥2 (105×), make every scan workload at K=4 scale positively, and deliver sharded find‑by parity without the --pool-workers flag. See docs/BENCHMARKS.md §14 + §14b + §14c + §14d.
• Analytical aggregates (TPC‑H Q1/Q6). Range‑pred narrowing on aggregate scans, single‑scan multi‑aggregate folding, a parallel hash aggregate (per‑worker HashMap partials + sorted‑BTreeMap merge) for large row counts, batched streaming overlap so workers start folding on row 1, and a WHERE filter compiled once per query into a closure that captures pre‑resolved field offsets + comparison ops + the AND/OR short‑circuit tree. Together these closed the gap vs Postgres from ~18× to 2.16× (Q1) and from ~123× to 3.09× (Q6). Q1 and Q6 remain the two workloads where Postgres still wins; closing the residual gap (now in the decode→update fold work, not WHERE evaluation) is a JIT‑aggregate follow‑up. See docs/BENCHMARKS.md §3f + §3g.

Headline numbers worth quoting (see docs/BENCHMARKS.md §1):

• Sharded point‑read get‑by‑id, K=8, N=16 workers: KesselDB 14.71M ops/sec (3.00× the 4.91M K=1 baseline; sub‑µs p50; K=16 → 16.24M)
• YCSB‑C reads, N=16: KesselDB 5.27M ops/s — 63.75× Postgres
• YCSB‑B mixed (95/5), N=16: KesselDB 573.6K ops/s — 7.26× Postgres
• sysbench OLTP write‑only, N=8: KesselDB 50.7K tx/s — 4.91× Postgres
• sysbench OLTP read‑only, N=16: KesselDB 30.6K tx/s — 6.02× Postgres
• sysbench OLTP read‑write, N=16: KesselDB 8.85K tx/s — 2.30× Postgres
• TPC‑H Q6, N=4: KesselDB 544.59 q/s — gap vs Postgres 3.09× (design floor ≥400 + stretch ≥500 both met)
• KesselDB wins 6 of 8 cross‑DB workloads vs Postgres (only TPC‑H Q1+Q6 remain losses)
• PG COPY FROM STDIN, 100K rows, single conn: KesselDB 51,840 rows/sec — 181.9× lift over the bulk‑apply baseline

Every figure is reproducible from the test suite / kessel-bench, and each query accelerator is guarded by a randomized equivalence oracle (the accelerated result is proven identical to a brute‑force scan). Full methodology, the single‑core/fsync/RTT scaling model, and order‑of‑magnitude projections for common cloud instance + storage configurations are in docs/PERFORMANCE.md.

Parquet capability matrix

The kessel-parquet crate is a from‑scratch, zero‑external‑dependency Parquet reader. Its capability surface, proven by hand‑derived KATs against published Apache spec text + by real pyarrow 24.0.0 round‑trip fixtures:

Still deferred (typed Unsupported at REFRESH with a precise error naming the follow‑on slice):

• Legacy LZ4 framing (codec id 5, deprecated Hadoop variant — modern LZ4_RAW codec id 7 is fully supported via SP149)
• 4‑deep nesting (List<List<List<List<T>>>> etc.) — would be SP147 if a real fixture demands it; all 3‑deep and below now supported (OBJ-2c-5 fully closed at SP146)
• DECIMAL precision > 38 (would need i256)
• Per‑page decompressed size > 256 MiB (SP151 lifted the 64 MiB historical cap; operators with known-trusted producers can lower or raise the cap via extract_with_cap up to the per-codec module ceiling)

The reader is feature‑gated through kessel-fetch's object-store feature; the default cargo build links no Parquet code at all and the kernel's deterministic state machine is unaffected.

Project status & maturity

KesselDB is a complete, functionally‑correct relational SQL database on a VSR‑safe, liveness‑tested, real multi‑node consensus core. Every named production‑readiness gate is met (functional completeness, crash recovery, VSR safety + adversarial‑partition liveness, multi‑node over sockets, full SQL over the cluster, exactly‑once + failover, auth/quotas/backpressure, hot backup + metrics). See the gate table in docs/STATUS.md.

Honest boundaries (documented, not hidden):

• Transport encryption (TLS) is an opt‑in cargo feature (--features tls, rustls) so the default build stays strictly zero‑dependency. Without it the wire is plaintext but token‑authenticated with a timing‑safe comparison (deploy behind a TLS proxy / private network). Hand‑rolling TLS would be irresponsible, hence the feature.

• HTTPS external sources are an opt‑in cargo feature (--features external-sources-tls, rustls + webpki‑roots) so the default build and plain --features external-sources remain zero‑new‑dependency and http://‑only. Enable this feature to register https:// endpoints; without it only plaintext HTTP is accepted.

• Object-store external sources (S3 / Azure Blob) are an opt‑in cargo feature (--features external-sources-objstore, which implies external-sources-tls and pulls rustls + webpki‑roots + the kessel-objstore crate). The default build and plain --features external-sources remain unaffected. Enable this feature to register s3:// or az:// endpoints; without it those URL schemes are rejected at CREATE with a clear message. Object-store requests are HTTPS-only with full webpki certificate verification and no bypass. FORMAT PARQUET for s3:///az:// is supported under this same feature; the kessel‑parquet crate has its own empty [dependencies] (the entire reader is hand‑written zero‑dep Rust). See the Parquet capability matrix below for the exact matrix of supported encodings / compressions / types / page versions.

• Cross‑shard transactions are implemented, deterministically (Calvin‑style), over real sockets — not blocking two‑phase commit. A deployment runs K independent VSR shard groups behind a router (rendezvous key→shard mapping). A cross‑shard Op::Txn is decomposed into per‑shard slices, durably totally ordered by a replicated sequencer group, then each shard applies its slice in that order: a deterministic decide → commit in which every shard’s verdict is a pure function of its durable state, so the global AND decision is recomputable by any router with no coordinator‑failure hole and no locks held across shards. It is atomic (a slice that would fail aborts the transaction on every shard), exactly‑once under client retry (stable (client,req) keying), and recoverable (a full ordered re‑drive after a router restart is idempotent). Single‑shard transactions stay on their shard’s own VSR group (serializable, fast path). Proven by a deterministic adversarial‑drive test composed on the seeded per‑group partition corpus, plus over‑sockets atomicity/abort/exactly‑once/recovery and concurrency tests. Balance‑guard helpers, destructive ALTER TABLE (DROP/RENAME COLUMN), DROP INDEX, DROP TABLE, and overflow‑blob GC are all implemented.

  Boundary (documented, not hidden): the router serializes cross‑shard commits to drive the global order; an async per‑shard pull‑drive is an efficiency follow‑up, not a correctness change. Cross‑shard transactions are point‑op batches (Create/Update/ Delete); cross‑shard scatter‑gather reads/SQL text routing is a separate, later concern from cross‑shard transactions.

Every claim in this repository is backed by the test suite (~2,700+ tests by default, more with the PostgreSQL and HTTP gateway features); the docs call out exactly what is proven versus roadmap. The four strategic‑tier items S1–S4 (TLA+/model‑checked safety, serializable MVCC/SI, Jepsen linearizability under partition, deterministic WASM UDFs) are all shipped — see docs/THESIS.md for the framing, and docs/STATUS.md for per‑slice records.

Documentation

Building & testing

cargo build                 # all kernel crates, zero external deps
cargo test --workspace      # ~2,700+ default tests (seeded partition/fault sim,
                            # Jepsen linearizability, MVCC TLA+ refinement,
                            # pyarrow Parquet round-trips, WASM-MVP KATs,
                            # 85-seed cross-shard K-invariance sweep)
cargo test --workspace --features pg-gateway                # adds the PostgreSQL gateway suite
cargo test --workspace --features pg-gateway,http-gateway,kessel-http-gateway/test-server   # full matrix
cargo run -p kessel-bench --release -- --help               # benchmarks

# Strategic-tier rigor artifacts:
cd kesseldb-tla/ && tlc -workers auto Replication.tla       # ≥528M states / depth 21 / 0 violations

Requires Rust stable 1.95+. No system libraries, no native build steps.

Contributing

Issues and PRs welcome. The project rule is simple and strict: every change is test‑driven, the full suite stays green, and documentation/claims never exceed what the tests prove. Each unit of work ships as one reviewed slice with its own spec under docs/superpowers/specs/.

License

MIT License — see LICENSE. © 2026 Ian Hassard.

Quick start

Five minutes from download to a real SQL query, including the PostgreSQL wire. Full details live in the Usage guide — this chapter is the README's quick-start section, kept verbatim so you don't need to dig.

For the README-side context (pitch, capability matrix, performance log links), see the Introduction. For the full operator manual including auth, quotas, clustering, backups, and every wire protocol, see Usage guide (full) §1–§13.

Download a prebuilt Linux binary

VER=v1.0.0   # see https://github.com/hassard0/KesselDB/releases for the latest
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/kesseldb-$VER-x86_64-unknown-linux-gnu \
  -o kesseldb && chmod +x kesseldb
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/kessel-$VER-x86_64-unknown-linux-gnu \
  -o kessel    && chmod +x kessel

The release workflow builds these with cargo build --release --features pg-gateway,http-gateway so the PostgreSQL, HTTP, and WebSocket gateways are wired in.

Or build from source

git clone https://github.com/hassard0/KesselDB && cd KesselDB
cargo build --release                                       # binary protocol only
cargo build --release --features pg-gateway,http-gateway    # all wire surfaces
cargo test  --workspace --release                           # 2018 default tests

Requires Rust stable 1.95+. No system libraries, no native build steps.

Start a node

# kesseldb [LISTEN_ADDR] [DATA_DIR]
./kesseldb 127.0.0.1:7878 ./data

# All wire surfaces (binary + HTTP + PG) on one node:
KESSELDB_TOKEN=mysecret \
KESSELDB_HTTP_ADDR=127.0.0.1:8080 \
KESSELDB_PG_ADDR=127.0.0.1:5432 \
  ./kesseldb 127.0.0.1:7878 ./data

Connect

# Binary protocol via the kessel CLI:
./kessel "CREATE TABLE acct (owner U32 NOT NULL, bal I64 NOT NULL)"
./kessel "INSERT INTO acct ID 1 (owner, bal) VALUES (100, 50)"
./kessel "SELECT SUM(bal) FROM acct WHERE owner = 100"     # => = 50

# HTTP/1.1 + JSON:
curl -s -X POST --data-binary 'SELECT * FROM acct' \
  -H 'Content-Type: text/plain' \
  -H 'Authorization: Bearer mysecret' \
  http://127.0.0.1:8080/v1/sql

# PostgreSQL wire (any libpq client):
PGPASSWORD=mysecret psql -h 127.0.0.1 -p 5432 -U test "SELECT SUM(bal) FROM acct"

Next steps: CLI · SQL surface · HTTP gateway · PostgreSQL wire · Running a cluster.

Install

KesselDB is pure Rust with no external dependencies in the kernel and no native build steps.

• Prebuilt Linux x86_64 binary — grab from the releases page. Each release ships a server binary (kesseldb), a CLI binary (kessel), a bundle tarball, and SHA256SUMS.
• Build from source —

  git clone https://github.com/hassard0/KesselDB && cd KesselDB
  cargo build --release                                       # default — binary protocol only
  cargo build --release --features pg-gateway,http-gateway    # all wire surfaces
  

  Requires Rust stable 1.95+.

Full install + build matrix: Usage guide (full) §1.

Usage guide (full)

This chapter is the full content of docs/USAGE.md in the repository — install, run, CLI, client API, SQL reference, clustering, auth, ops, external sources, every wire protocol, troubleshooting. The smaller per-topic chapters that follow (CLI, SQL surface, HTTP, WebSocket, PostgreSQL wire, External sources) are deep-link landing pages pointing into specific sections of this guide.

KesselDB — Usage Guide

Everything you need to install, run, query, cluster, secure, and operate KesselDB. Every feature described here is covered by the test suite.

• 1. Install & build
• 2. Run a server
• 2b. The kessel command-line client
• 3. The client library
• 4. SQL reference
• 5. The data model
• 6. Transactions
• 7. Running a cluster
• 7b. Sharded deployment & cross-shard transactions
• 7c. External sources (JSON/CSV over HTTP)
• 7d. Paginated & NDJSON sources
• 7e. Object-store sources (S3 / Azure Blob)
• 7f. FORMAT PARQUET for object-store sources
• 8. Authentication, quotas & backpressure
• 9. PostgreSQL clients (psql, pgcli, JDBC, psycopg, pgx, …)
• 10. HTTP gateway (and WebSocket)
• 11. Backup & monitoring
• 12. Wire protocol
• 13. Troubleshooting

1. Install & build

Option A — download a prebuilt binary (Linux x86_64)

KesselDB ships prebuilt server (kesseldb) and CLI (kessel) binaries for x86_64-unknown-linux-gnu on the GitHub Releases page. Each release is built from cargo build --release --features pg-gateway,http-gateway, so the PostgreSQL, HTTP/1.1, and WebSocket gateways are wired in.

VER=v2.0.0   # see the releases page for the latest
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/kesseldb-$VER-x86_64-unknown-linux-gnu \
  -o kesseldb && chmod +x kesseldb
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/kessel-$VER-x86_64-unknown-linux-gnu \
  -o kessel    && chmod +x kessel

# or grab the bundle (server + CLI + README + USAGE + LICENSE):
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/kesseldb-$VER-x86_64-unknown-linux-gnu.tar.gz \
  | tar xz

# SHA-256 checksums are published alongside the binaries:
curl -L https://github.com/hassard0/KesselDB/releases/download/$VER/SHA256SUMS -o SHA256SUMS
sha256sum -c SHA256SUMS --ignore-missing

Option B — run from the official Docker image

A pre-published multi-arch image (linux/amd64 + linux/arm64) is pushed to GitHub Container Registry on every v* release. The image is the existing --features pg-gateway,http-gateway server, runs as a non-root kessel:1100 UID, and exposes all three wire surfaces.

# Pull and run, mounting a host data dir + a one-token auth surface.
docker run --rm \
  -p 6532:6532 -p 6533:6533 -p 5432:5432 \
  -v $PWD/kesseldb-data:/data \
  -e KESSELDB_TOKEN=changeme \
  ghcr.io/hassard0/kesseldb:latest

# From another shell, the bare kessel CLI works exactly like local:
kessel --addr 127.0.0.1:6532 --token changeme \
  'CREATE TABLE acct (owner U32 NOT NULL, bal I64 NOT NULL)'
# or via the HTTP gateway on :6533, or psql on :5432.

Default ports inside the container: 6532 binary, 6533 HTTP+WS, 5432 PostgreSQL. Persist the data dir with -v <host>:/data. The image is rebuilt from Dockerfile at the repo root — see that file for the two-stage layout (rust:1-slim builder → debian-bookworm-slim runtime, ~77 MiB stripped).

Option C — build from source

KesselDB is pure Rust with no external dependencies in the kernel and no native build steps.

git clone https://github.com/hassard0/KesselDB && cd KesselDB
cargo build --release                                # default — binary protocol only, no gateway code linked
cargo build --release --features pg-gateway,http-gateway   # all wire surfaces
cargo test --workspace                               # 2442 default tests
cargo test --workspace --features pg-gateway         # 2470 (adds SP-PG + SP-PG-CAT + SP-PG-EXTQ V1 + V2 hardening + SP-PG-COPY V1)
cargo test --workspace --features pg-gateway,http-gateway,kessel-http-gateway/test-server   # 2503 — full matrix

Requires Rust stable 1.95+.

Workspace crates (use the ones you need as path/library deps):

2. Run a server

The kesseldb binary runs a single, open node (no auth) — the simplest way to get going:

# kesseldb [LISTEN_ADDR] [DATA_DIR]
cargo run --release --bin kesseldb -- 127.0.0.1:7878 ./data
# defaults: 127.0.0.1:7878  ./kesseldb-data

The data directory holds the WAL, SSTables and manifest. Stop and restart the process and it recovers from the WAL automatically (crash‑safe, torn‑tail handled).

For authentication, quotas, or a multi‑node cluster you compose the kesseldb-server library API — see §7 and §8.

2b. The kessel command-line client

Query KesselDB without writing any code — the fastest path for humans, scripts, ops, and agents.

# one-shot (exit 0 = success, 1 = statement/connection error, 2 = bad usage)
cargo run -q -p kessel-client --bin kessel -- "CREATE TABLE t (v U64 NOT NULL)"
cargo run -q -p kessel-client --bin kessel -- "INSERT INTO t ID 1 (v) VALUES (42)"
cargo run -q -p kessel-client --bin kessel -- "SELECT SUM(v) FROM t"   # => = 42

# a whole-row SELECT prints a real aligned table (no DESCRIBE needed):
#   owner | bal
#   ------+----
#   100   | 50
#   (1 row)
kessel "SELECT * FROM t ID 1"
kessel "SELECT * FROM t WHERE owner = 100"
kessel "SELECT owner, bal FROM acct"      # projections render too
kessel "SELECT * FROM a JOIN b ON a.x = b.y"   # JOINs render too (self-describing)
kessel "SELECT a.n, b.t FROM a JOIN b ON a.id = b.aid WHERE b.t = 'x'"  # filtered joins (JOIN + WHERE)
kessel "SELECT a.n, b.t FROM a LEFT JOIN b ON a.id = b.aid"  # LEFT [OUTER] JOIN — unmatched a-rows keep b.* = NULL
kessel "SELECT a.n, b.t FROM a RIGHT JOIN b ON a.id = b.aid" # RIGHT [OUTER] JOIN — unmatched b-rows keep a.* = NULL (column order stays a.*,b.*)
kessel "SELECT a.n, b.t FROM a FULL JOIN b ON a.id = b.aid"  # FULL [OUTER] JOIN — matched + unmatched on BOTH sides, no dup (SP-PG-SQL-RIGHT-FULL-JOIN)
kessel "SELECT a.n, b.t FROM a JOIN b ON a.id = b.aid ORDER BY b.t LIMIT 20 OFFSET 40"  # paginated join (ORDER BY + LIMIT/OFFSET)
kessel "SELECT a.n, COUNT(b.id) FROM a JOIN b ON a.id = b.aid GROUP BY a.n"  # grouped aggregate over a join (count related per parent)
kessel "SELECT users.name, posts.title, comments.body FROM users JOIN posts ON users.id = posts.user_id JOIN comments ON posts.id = comments.post_id"  # chained 3-way INNER join (SP-PG-SQL-MULTI-JOIN)
kessel "SELECT u.name, p.title FROM users u JOIN posts p ON u.id = p.user_id"  # table aliases resolve everywhere (SP-PG-SQL-JOIN-ALIAS) — the SQLAlchemy/Django form

# pipe a .sql file (lines starting with # or -- are comments; blanks ignored)
cat schema.sql | cargo run -q -p kessel-client --bin kessel

# machine-readable: one JSON object per statement (ideal for agents)
kessel --json "SELECT * FROM t"
#   {"status":"ok","rows":[{"v":42}]}
kessel --json "SELECT SUM(v) FROM t"      # {"status":"ok","value":42}
kessel --json "DESCRIBE t"                # {"status":"ok","table":"t","columns":[…]}
kessel --json "SELECT * FROM nope"        # {"status":"error","message":"…"}  (exit 1)

# DESCRIBE / \d render a readable schema (text mode):
#   table t
#   column | type | null
#   -------+------+-----
#   v      | U64  | NO
kessel "DESCRIBE t"

# interactive shell (TTY): a `kessel>` prompt
cargo run -q -p kessel-client --bin kessel
#   \?  \h  \help      list shell commands
#   \d <table>         describe a table
#   \timing            toggle per-statement timing
#   \q  quit  exit     leave

# remote / authenticated
kessel --addr 10.0.0.1:7878 --token s3cret "SELECT * FROM t ID 1"

kessel [--addr HOST:PORT] [--token TOKEN] [--json] ["SQL"] — default address 127.0.0.1:7878. With no SQL argument it reads statements from stdin (one per line). The exit code is reliable (0 ok, 1 statement/connection error, 2 bad usage) and --json emits one stable object per statement, so an agent or script can branch on success without parsing prose. (After cargo build --release the binary is target/release/kessel.)

3. The client library

kessel-client is a minimal blocking client. Add it as a path dependency, or copy the wire protocol (§10) into any language.

Python — a dependency-free, stdlib-only reference client ships at clients/python/kesseldb.py:

from kesseldb import connect
db = connect("127.0.0.1:7878")            # connect(addr, token=b"..") for auth
db.sql("CREATE TABLE acct (owner U32 NOT NULL, bal I64 NOT NULL)")
db.sql("INSERT INTO acct ID 1 (owner, bal) VALUES (100, 50)")
print(db.sql("SELECT SUM(bal) FROM acct WHERE owner = 100").value)  # 50
db.close()

Or one-shot: python clients/python/kesseldb.py "SELECT …" [--addr H:P] [--token T] (exit 0 ok / 1 error / 2 usage). It is a faithful, tested implementation of §10 — the template for an SDK in any language.

Single node

#![allow(unused)]
fn main() {
use kessel_client::Client;
use kessel_proto::{Op, ObjectId, OpResult};

let mut db = Client::connect("127.0.0.1:7878")?;

// SQL (compiled server‑side against the live catalog):
db.sql("CREATE TABLE acct (owner U32 NOT NULL, bal I64 NOT NULL)")?;
db.sql("INSERT INTO acct ID 1 (owner, bal) VALUES (100, 50)")?;
let r = db.sql("SELECT SUM(bal) FROM acct WHERE owner = 100")?;

// Low‑level ops (no SQL parse), if you want them:
db.call(&Op::Create { type_id: 1, id: ObjectId::from_u128(2), record: vec![/* codec bytes */] })?;
db.call(&Op::GetById { type_id: 1, id: ObjectId::from_u128(2) })?;
}

OpResult variants you will see: Ok, Got(bytes), Exists, NotFound, TypeCreated(id), Constraint(msg), SchemaError(msg), plus the transport signals Unavailable (not the primary — try another node) and Unauthorized.

Authenticated connection

#![allow(unused)]
fn main() {
let mut db = Client::connect_authed("127.0.0.1:7878", b"my-shared-secret")?;
}

Cluster client (automatic failover, exactly‑once)

#![allow(unused)]
fn main() {
use kessel_client::ClusterClient;

let mut db = ClusterClient::new(vec![
    "10.0.0.1:7878".into(), "10.0.0.2:7878".into(), "10.0.0.3:7878".into(),
]);                                  // .with_token(b"secret".to_vec()) if authed

db.call(&op)?; // finds the primary, retries the *same* (client,req) on
               // Unavailable/connection loss — never double‑applies
}

ClusterClient holds a stable session id and a monotonic request number, so a retry after a primary change returns the original committed reply rather than re‑executing.

Embedded — KesselDB inside your Rust process

Skip the network round-trip entirely: depend on kesseldb-server directly and call the engine in-process. Read paths take the SP‑Perf‑A bypass under an RwLock::read() (sub‑µs latency); writes still serialise through the engine thread's deterministic apply.

#![allow(unused)]
fn main() {
use kesseldb_server::{spawn_engine_cfg, ServerConfig};
use kessel_proto::OpResult;

let cfg = ServerConfig { read_workers: Some(0), ..Default::default() };
let engine = spawn_engine_cfg("./kesseldb-data", &cfg)?;

// SQL fast path — same compile + apply as a wire `[0xFE] ++ sql` frame,
// minus the socket.
engine.sql("CREATE TABLE acct (owner U32 NOT NULL, bal I64 NOT NULL)");
engine.sql("INSERT INTO acct ID 1 (owner, bal) VALUES (100, 50)");
match engine.sql("SELECT SUM(bal) FROM acct WHERE owner = 100") {
    OpResult::Got(b) => println!("sum = {}", i128::from_le_bytes(b[..16].try_into().unwrap())),
    other            => panic!("{other:?}"),
}

// Hot consistent backup — copies the data dir while no apply is in flight.
engine.snapshot("./kesseldb-data.snap")?;
}

A complete walkthrough — typed Op fast path, kessel_codec::encode round-trip, snapshot — lives at crates/kesseldb-server/examples/embedded.rs. Run it from the repo root:

cargo run --release --example embedded -p kesseldb-server

4. SQL reference

Compiled server‑side against the live catalog. Supported surface (each item is tested):

DDL

CREATE TABLE <t> (<col> <TYPE> [NOT NULL], ...)
CREATE TABLE <child> (..., <fkcol> <TYPE>,   -- FOREIGN KEY is ENFORCED:
  FOREIGN KEY(<fkcol>) REFERENCES <parent>(id)  --  a non-NULL <fkcol> with no
    [ON DELETE NO ACTION|RESTRICT|CASCADE|SET NULL|SET DEFAULT])  -- matching parent
                                            --  row is rejected (SQLSTATE 23503);
                                            --  NULL is allowed. Inline form
                                            --  `<fkcol> <TYPE> REFERENCES <parent>(id)`
                                            --  works too. Create the parent FIRST.
ALTER TABLE <t> ADD [COLUMN] <c> <TYPE> [NOT NULL]  -- online, no lock; old rows: NULL
DROP TABLE <t>                              -- removes rows, indexes & the type
                                            -- (refused if an FK still points at it)
CREATE INDEX        ON <t> (<col>)          -- equality index
CREATE UNIQUE INDEX ON <t> (<col>)          -- unique constraint + index
CREATE RANGE  INDEX ON <t> (<col>)          -- order‑preserving (range scans)
CREATE INDEX        ON <t> (<c1>, <c2>)     -- composite
DESCRIBE <t>                                -- returns the table definition
EXPLAIN <stmt>                              -- prints the plan, runs nothing

Column types: U8 U16 U32 U64, I8 I16 I32 I64, BYTES, BOOL.

DML

INSERT INTO <t> ID <n> (<cols>) VALUES (<vals>)            -- legacy single-row
INSERT INTO <t> (id, <cols>) VALUES (<v>)[, (<v>)]*       -- Postgres-shaped;
                                                          -- multi-row = 1 atomic op
INSERT INTO <t> (<cols>) VALUES (<v>) [RETURNING <c>,..]  -- autoincrement: omit the
                                                          --   BIGSERIAL PK id; the
                                                          --   engine assigns it
                                                          --   deterministically and
                                                          --   RETURNING reads it back
UPDATE <t> SET <col> = <val> [, ...] WHERE <expr> [RETURNING <c>,..|*]
                                                 -- multi-row: every row matching
                                                 --   the WHERE predicate is mutated
                                                 --   atomically (one logical txn)
DELETE FROM <t> WHERE <expr> [RETURNING <c>,..|*]
                                                 -- multi-row: every match removed
                                                 --   atomically; RETURNING yields
                                                 --   the deleted rows
UPDATE <t> ID <n> SET <col> = <val> [, ...]      -- legacy by-id read‑modify‑write
INSERT INTO <t> (id, <c>) VALUES (<n>, NULL)     -- explicit SQL NULL for a
                                                 --   nullable column

NULL semantics. A nullable column that is OMITTED from an INSERT's column list, or given an explicit NULL value (INSERT INTO t (id, c) VALUES (1, NULL)), is stored as a true SQL NULL (the row's null-bitmap bit is set) and reads back as a real NULL over the PG wire — psycopg2 None, NOT 0 or an empty string — for BOTH SELECT * and a projection-list SELECT c FROM t. This holds for every column kind (integer, text/CHAR, numeric). Omitting a NOT NULL column with no DEFAULT is rejected; an explicit NULL on a NOT NULL column or on the id primary key is also rejected. A defaulted / BIGSERIAL PK column keeps its assigned value (it is never turned into NULL).

General-WHERE UPDATE/DELETE. The WHERE clause accepts the SAME predicate grammar as SELECT (=, !=, <, <=, >, >=, AND/ OR/NOT, IN, BETWEEN, IS [NOT] NULL), so UPDATE users SET active = 0 WHERE last_login < $1 and DELETE FROM t WHERE status = 'expired' mutate every matching row in ONE atomic statement. The server resolves the matching rows (a deterministic scan), then applies a single replicated transaction of per-row mutations — so a constraint violation on ANY row (e.g. a UNIQUE collision) rolls the WHOLE statement back (zero rows applied). The CommandComplete tag carries the real affected count (UPDATE N / DELETE N). RETURNING <cols> / RETURNING * returns the affected rows (post-mutation for UPDATE, the deleted rows for DELETE) — ORMs use UPDATE … WHERE id = $1 RETURNING * for optimistic concurrency. An unguarded table-wide UPDATE/DELETE (no WHERE) is rejected in V1 (a footgun guard). V1 does a full predicate scan (no index narrowing yet — SP-PG-SQL-DML-PLAN); UPDATE … FROM / DELETE … USING joins, correlated subqueries, and SET col = col + expr are named follow-ups.

For a CREATE TABLE <t> (id BIGSERIAL PRIMARY KEY, ...), an INSERT that OMITS id autoincrements: the engine assigns the next per-table sequence value (deterministic + replicated — the counter lives in the state digest, advanced only on the apply thread). INSERT … RETURNING id returns the assigned id (the SQLAlchemy/ORM autoincrement default).

The SQL-standard autoincrement spelling id bigint GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( START WITH n INCREMENT BY n ) ] is an alias for the same deterministic SERIAL autoincrement — this is the DDL Django 6's default BigAutoField emits (it renders IDENTITY, not BIGSERIAL). The optional sequence-options group is parsed-and-ignored in V1 (the counter always starts at 1, increments by 1; custom start/ increment is the named follow-up SP-PG-IDENTITY-SEQOPTS).

Queries

SELECT * FROM <t> ID <n>                         -- O(1) primary‑key fetch
SELECT * FROM <t> [WHERE <expr>]                 -- =, !=, <, <=, >, >=, AND/OR/NOT,
                                                 --   col IN (a,b,..), col BETWEEN lo AND hi
                                                 --   col IS [NOT] NULL, col [NOT] LIKE 'pat%' (NOT IN / NOT BETWEEN too)
SELECT <c1>, <c2> FROM <t> [WHERE ...]           -- projection
SELECT DISTINCT <c1>[, <c2> …] FROM <t> [WHERE ...] [ORDER BY …]  -- dedup result rows (SP-PG-SQL-DISTINCT)
SELECT DISTINCT * FROM <t> [WHERE ...]           -- dedup whole rows; NULL is not distinct from NULL
                                                 --   (DISTINCT ON (…), DISTINCT over JOIN/aggregate/GROUP BY are follow-ups)
SELECT COUNT(*) | SUM(c) | MIN(c) | MAX(c) | AVG(c) FROM <t> [WHERE ...]  -- single scalar aggregate (SP-PG-SQL-AGG-ALIAS-RENDER)
SELECT <g1>[, <g2> …], <AGG>( * | <c> ) [AS alias] [, <AGG>(…) …] FROM <t> [WHERE …]  -- plain GROUP BY group-aggregate (SP-PG-SQL-PLAIN-GROUP-RENDER)
       GROUP BY <g1>[, <g2> …]                          --   COUNT/SUM/MIN/MAX/AVG per group; one row per (g1[,g2,…]); multi-column (SP-PG-SQL-GROUP-MULTI-COL)
       [HAVING <AGG>(...) <cmp> <literal>]              --   filter groups by an aggregate (SP-PG-SQL-HAVING)
       [ORDER BY <agg|alias|pos|g> [ASC|DESC]] [LIMIT n] [OFFSET n]  --   engine sorts + windows groups (SP-PG-SQL-GROUP-SORT-LIMIT)
SELECT * FROM <t> [WHERE ...] ORDER BY <col> [DESC] [OFFSET n] [LIMIT n]
SELECT <c> FROM <t> WHERE <col> IN     (SELECT <c2> FROM <u> [WHERE …])  -- non-correlated WHERE subquery (SP-PG-SQL-SUBQUERY-WHERE)
SELECT <c> FROM <t> WHERE <col> NOT IN (SELECT <c2> FROM <u> [WHERE …])  --   IN / NOT IN over a 1-column inner SELECT
SELECT <c> FROM <t> WHERE <col> = (SELECT MAX(<c2>) FROM <u>)            --   scalar subquery: = <> < <= > >= over a 1-row/1-col inner
SELECT <proj> FROM <a> [[AS] <a1>] [INNER|LEFT|RIGHT|FULL [OUTER]] JOIN <b> [[AS] <a2>] ON <a1.x> = <a2.y>  -- equi‑join (table aliases OK, SP-PG-SQL-JOIN-ALIAS)
       [JOIN <c> [[AS] <a3>] ON <a1|a2.x> = <a3.y> [JOIN <d> ON …]]  --   chained N-way INNER joins, 3+ tables (SP-PG-SQL-MULTI-JOIN)
       [WHERE <pred over any joined table's cols>]       --   INNER (default) / LEFT / RIGHT / FULL; filtered (qualified cols, AND/OR/…)
       [ORDER BY <t.c> [ASC|DESC]] [LIMIT n] [OFFSET m]  --   paginate the (sorted) join
SELECT <a.g1>[, <b.g2> …], <AGG>( * | <a.c | b.c> ) [AS alias] [, <AGG>(…) …]  -- grouped aggregate over a join
       FROM <a> [INNER|LEFT|RIGHT|FULL [OUTER]] JOIN <b> ON <a.x> = <b.y> [WHERE …]
       GROUP BY <a.g1>[, <b.g2> …]                      --   COUNT/SUM/MIN/MAX/AVG per group; one row per (g1[,g2,…]); multi-column (SP-PG-SQL-GROUP-MULTI-COL)
       [HAVING <AGG>(...) <cmp> <literal>]              --   filter groups after aggregation (SP-PG-SQL-HAVING)

A bare JOIN (or INNER JOIN) is an INNER equi‑join. The full outer-join matrix is supported on a binary (two-table) join (SP-PG-SQL-OUTER-JOIN + SP-PG-SQL-RIGHT-FULL-JOIN):

The combined column ORDER is ALWAYS a.* then b.*, regardless of flavour (you wrote FROM a … JOIN b, so SELECT a.x, b.y keeps resolving) — a RIGHT join NULLs the a.* side, not the column order. NULL-filled columns read back as SQL NULL (Python None in psycopg2). A WHERE predicate on the NULL-filled side of an outer join drops the unmatched rows — standard PostgreSQL semantics. This is the ORM pattern for optional relationships (SQLAlchemy isouter=True / full=True). RIGHT/FULL on a binary join compose with WHERE / ORDER BY / LIMIT / OFFSET / GROUP BY / aliases exactly like LEFT. RIGHT/FULL mixed into a 3+ table CHAIN is a named follow-up (rejected with a clear error; INNER chains keep working).

Chained N-way joins (SP-PG-SQL-MULTI-JOIN) — 3+ tables joined in one query, the everyday "row + its parent + its grandparent" shape:

SELECT users.name, posts.title, comments.body
  FROM users JOIN posts ON users.id = posts.user_id
             JOIN comments ON posts.id = comments.post_id
  WHERE users.id = 1;

Each additional JOIN <table> ON <a.x> = <table.y> segment INNER equi-joins the running combined row set against the next table; the combined schema widens by that table's columns each step (named <table>.<col>). SELECT * returns every column of every joined table; WHERE / ORDER BY / LIMIT / OFFSET apply over the full combined schema. V1 is INNER chains only — mixing LEFT/RIGHT/FULL into a chain, or GROUP BY over a chain, are named follow-ups (rejected with a clear error).

Table aliases (SP-PG-SQL-JOIN-ALIAS) — each table in the FROM/JOIN clause may carry an optional [AS] <alias> (FROM users u, FROM users AS u), and the alias resolves in EVERY qualifier — projection, ON, WHERE, ORDER BY, GROUP BY — for both binary and multi-table joins:

SELECT u.name, p.title, c.body
  FROM users u JOIN posts p ON u.id = p.user_id
               JOIN comments c ON p.id = c.post_id
  WHERE u.id = 1 ORDER BY p.title;

This is the form SQLAlchemy / Django / Rails emit. The alias is resolved to the full table name in the SQL layer, so an aliased join compiles to the IDENTICAL wire op as its full-table-name twin — full-name qualifiers (users.name) keep working unchanged (back-compat). A duplicate/ambiguous alias, an alias that shadows another table's name, or an unknown qualifier is a clean error. A self-join joining the SAME table under two aliases (FROM users a JOIN users b) is a named follow-up (SP-PG-SQL-SELF-JOIN), rejected to avoid same-name ambiguity in the combined schema.

ORDER BY <qualified col> sorts the combined join rows by ONE column from either table (ASC default / DESC); LIMIT + OFFSET then paginate the sorted result — the ubiquitous paginated-list-view shape (… ORDER BY b.created LIMIT 20 OFFSET 40). For a LEFT join, an unmatched right (b.*) NULL sort value orders NULLS LAST for ASC / NULLS FIRST for DESC (PostgreSQL's default).

Plain (single-table) GROUP BY (SP-PG-SQL-PLAIN-GROUP-RENDER) is the everyday analytics / ORM aggregation — "count (or sum / avg / …) rows per category":

SELECT category, COUNT(*) FROM products GROUP BY category;
SELECT category, COUNT(*) AS n, SUM(price), AVG(price), MIN(price), MAX(price)
  FROM products GROUP BY category HAVING COUNT(*) > 1;

returns one row per group (group_key, agg_value+), groups in ascending group-key order, rendered over the PG wire. The group key column is typed from the table schema (int key → int4/int8, text key → text); aggregate columns are typed COUNT/SUM → int8, AVG → numeric, MIN/MAX → the source column's type. An unaliased aggregate gets PostgreSQL's default name (count/sum/avg/min/ max). HAVING (SP-PG-SQL-HAVING) filters the groups. V1 caveat: a trailing ORDER BY … LIMIT … OFFSET … on a plain GROUP BY is now applied by the engine (SP-PG-SQL-GROUP-SORT-LIMIT). The ORDER BY target may be a projected aggregate (by alias ORDER BY n, by 1-based position ORDER BY 2, or by the expression ORDER BY COUNT(*)) or the group key column (ORDER BY g / ORDER BY 1); DESC reverses, ties break by ascending group key, and LIMIT/OFFSET window AFTER the sort. HAVING filters BEFORE the sort, so the pipeline is filter → sort → offset → limit — making top-N-per-group analytics (… GROUP BY g ORDER BY COUNT(*) DESC LIMIT 5) work. The new fields are additive + marker-guarded: a query with no ORDER BY/LIMIT/OFFSET produces byte-identical Op frames to before, so the determinism oracles are untouched. ORDER BY over a JOIN group-aggregate remains the separate follow-up SP-PG-SQL-JOIN-AGG-ORDERBY-AGG.

Multi-column GROUP BY (SP-PG-SQL-GROUP-MULTI-COL) — group by SEVERAL columns to form a COMPOSITE group key, the bread-and-butter cross-tab analytics query:

SELECT region, category, COUNT(*), SUM(amount)
  FROM sales GROUP BY region, category;

returns one row per DISTINCT (region, category) tuple with the per-group aggregates. Every non-aggregate column in the SELECT list must appear in GROUP BY (PostgreSQL semantics); a GROUP BY column that isn't in the table / combined schema is a clean error. Columns may be bare (category), qualified (t.category), or aliased (u.category, resolved via the join alias map). It composes with HAVING (filter composite groups), ORDER BY (by any aggregate or the FIRST group column) and LIMIT/OFFSET — so GROUP BY region, category ORDER BY COUNT(*) DESC LIMIT 10 is top-N over composite groups. Works on a plain single-table GROUP BY AND over a binary join. The extra columns are additive + marker-guarded on the wire: a single-column GROUP BY produces byte-identical Op frames AND a byte-identical result stream to before, so the whole existing aggregate surface — and the determinism oracles — are untouched. Multi-column GROUP BY over a 3+ table chain is a named follow-up.

GROUP BY <a.g> + one or more aggregates over a join is the dashboard / reporting query — "count (or sum / …) the related rows per parent":

SELECT a.name, COUNT(b.id) FROM a JOIN b ON a.id = b.aid GROUP BY a.name

returns one row per group (group_key, agg_value+), groups in ascending group-key order. COUNT(*) is the group size; COUNT(b.id) counts the non-NULL b.id values in the group; SUM/MIN/MAX/AVG(<col>) fold the non-NULL numeric values. For a LEFT JOIN, an unmatched parent (all b.* = NULL) makes COUNT(b.id) = 0 but COUNT(*) = 1 — the classic LEFT-JOIN-COUNT gotcha, matching PostgreSQL exactly. The group + aggregate columns may come from either table; qualify an aggregate arg (COUNT(b.id)) when the column name exists in both tables. Multi-column GROUP BY over a binary join is supported (GROUP BY a.region, b.category, SP-PG-SQL-GROUP-MULTI-COL); a 3+-table join-agg remains a named follow-up.

HAVING <AGG>(...) <cmp> <literal> (SP-PG-SQL-HAVING) filters the GROUPS after aggregation — the "only show parents with more than N children" report:

SELECT a.name, COUNT(b.id) FROM a JOIN b ON a.id = b.aid
GROUP BY a.name HAVING COUNT(b.id) > 2

returns only the groups whose aggregate satisfies the predicate (here, parents with 3+ related rows). The HAVING aggregate MUST be one of the SELECT aggregates (matched by function + arg); <cmp> is any of > >= < <= = <> != and the RHS is an integer/numeric literal (negative allowed). HAVING composes with ORDER BY / LIMIT (the groups are filtered before paging) and works on both the plain GROUP BY and the over-JOIN forms. (V1: the HAVING aggregate must appear in the projection — a HAVING over an aggregate not selected, or over the group key, is a named follow-up.)

WHERE supports AND/OR/NOT, all of = != < <= > >=, and IN/BETWEEN (incl. NOT IN/NOT BETWEEN). SELECT * returns length‑prefixed record blobs; use DESCRIBE <t> to decode them against the schema (the client decodes the wire schema for you).

Non-correlated WHERE subqueries (SP-PG-SQL-SUBQUERY-WHERE) — a WHERE predicate may compare a column against the result of an inner SELECT:

SELECT name FROM users WHERE id IN (SELECT user_id FROM orders WHERE total > 100);
SELECT name FROM users WHERE id NOT IN (SELECT user_id FROM banned);
SELECT name FROM products WHERE price = (SELECT MAX(price) FROM products);  -- scalar

• col IN (subquery) / col NOT IN (subquery) — the inner SELECT must project exactly ONE column; the outer row matches if its col is (not) among the inner values.
• Scalar col <op> (subquery) for = <> != < <= > >= — the inner must project one column and yield AT MOST one row. Zero rows → the scalar is NULL → the comparison is NULL → the outer returns no rows.

The inner SELECT runs FIRST through the normal engine path, so it may itself use WHERE / aggregates (e.g. MAX(price)) — any SELECT shape that already works is a valid inner query. Spliced values are quoted by type (integers bare, text single-quoted with ' doubled). Empty inner result: IN (∅) returns no rows; NOT IN (∅) returns every (non-NULL) outer row — note a NULL-valued col is NOT returned by NOT IN (∅) (a documented V1 edge vs PostgreSQL's NULL-row handling). The inner projecting ≠ 1 column, or a scalar subquery returning > 1 row, is a clean error (never silently-wrong rows). V1 scope: ONE subquery per WHERE, NON-correlated only. Correlated subqueries (inner references an outer column), EXISTS/NOT EXISTS, subqueries in FROM (derived tables), subqueries in the SELECT list, and multiple subqueries per WHERE are named follow-ups.

Over the PostgreSQL wire, a single scalar aggregate SELECT COUNT(*) | SUM(c) | MIN(c) | MAX(c) | AVG(c) [AS alias] FROM <t> renders as one row, one column: the column is named by the AS alias when present, else the lowercase function name (count/sum/…), matching PostgreSQL's default output naming. This is what Django's .count(), .exists(), and .aggregate() emit (SELECT COUNT(*) AS "__count" FROM "t"). The grouped / multi-aggregate wire render is the named follow-up SP-PG-AGG-MULTI-RENDER.

Note: rows carry an explicit caller‑supplied ID (a 128‑bit key). There is no auto‑increment — the engine never generates ids, because that would introduce non‑determinism into the replicated state machine. Generate ids in your application (UUID, snowflake, etc.).

5. The data model

• Tables are runtime‑defined (CREATE TABLE) and can be altered online (add field) without downtime.
• Records are fixed‑width per the schema; variable‑length values use an overflow store transparently.
• Constraints: NOT NULL, UNIQUE, foreign keys — ENFORCED whether declared via CREATE TABLE … FOREIGN KEY(col) REFERENCES … DDL or ALTER/engine op; a non-NULL FK with no matching parent row is rejected (SQLSTATE 23503), a NULL FK is allowed, and ON DELETE NO ACTION | RESTRICT | CASCADE | SET NULL | SET DEFAULT are honored. CHECK (a deterministic, gas‑bounded expression program).
• Triggers: before‑write programs that may mutate or reject a row — same zero‑dep deterministic VM as CHECK.
• Indexes: equality, unique, order‑preserving (range), and composite.

Everything is applied through one deterministic state machine, so a given sequence of operations always produces the same state and the same content digest on every replica.

6. Transactions

SQL (single-node server) — BEGIN buffers subsequent statements; COMMIT applies them as one atomic unit; ROLLBACK discards them:

BEGIN;
INSERT INTO acct ID 1 (owner, bal) VALUES (100, 50);
INSERT INTO acct ID 2 (owner, bal) VALUES (100, 999);
COMMIT;          -- both rows land atomically; any failure aborts ALL

printf 'BEGIN\nINSERT INTO acct ID 9 (owner,bal) VALUES (1,1)\nCOMMIT\n' | kessel

A failing statement (e.g. a duplicate id) makes COMMIT fail and rolls back every statement in the transaction; the connection stays usable. COMMIT/ROLLBACK without BEGIN is a clean error. A by-id UPDATE composes inside a transaction (it lowers to the deterministic replicated Op::UpdateSet), and read-your-writes holds for writes within the batch (a later statement sees an earlier one's effect). A general-WHERE UPDATE/DELETE inside an explicit BEGIN/COMMIT is rejected in V1 (SP-PG-SQL-DML-IN-TXN) — its matched-id set would need to be resolved against the mid-transaction overlay; run it as a standalone auto-commit statement (which is itself atomic).

Model boundary (by design, not a TODO): a KesselDB transaction is an atomic, non-interactive write batch — serializable by construction. A SELECT/DESCRIBE/EXPLAIN inside BEGIN/COMMIT is rejected with a clear error: returning interactive read-your-writes mid-transaction would require holding the single engine overlay across client round-trips, serializing the whole engine — a deliberate non-goal. Run reads outside the transaction. UPDATE … SET col = NULL inside a transaction is the one unsupported write form (clear error; works outside a txn). Transactions are per-connection and single-node (the cluster front doesn't intercept the keywords — use op-level Op::Txn there).

Op level (works everywhere, incl. the cluster) — atomic, all‑or‑nothing, replicated as a single operation:

#![allow(unused)]
fn main() {
use kessel_proto::Op;
db.call(&Op::Txn { ops: vec![
    Op::Create { type_id: 1, id: a, record: ra },
    Op::Create { type_id: 1, id: b, record: rb },
]})?; // both apply, or neither — any failure rolls the whole batch back
}

If any inner op fails a constraint, the entire transaction is rejected with no visible side effects.

7. Running a cluster

A cluster is composed from the kesseldb-server library. Each node runs the deterministic engine wrapped in a VSR replica; nodes talk over TCP.

#![allow(unused)]
fn main() {
use kesseldb_server::cluster::{spawn_node, serve_clients};
use std::net::TcpListener;
use std::sync::Arc;

// Peer addresses, indexed by node id; all nodes share the same list.
let peers = vec![/* SocketAddr per node */];
let peer_listener = TcpListener::bind(my_peer_addr)?;       // this node's VSR socket
let node = Arc::new(spawn_node(my_idx, peer_listener, peers, "./data".into())?);

// Expose the ordinary client protocol for apps:
let client_listener = TcpListener::bind(my_client_addr)?;
serve_clients(client_listener, node.clone());
}

Properties (all tested, including a seeded adversarial partition corpus):

• Safety: a committed, client‑acknowledged operation is never lost across a view change.
• Liveness: once a quorum can communicate again, the cluster completes outstanding work and every replica reconverges to an identical digest.
• Exactly‑once: any node serves a committed (client, req) from its replicated client table; ClusterClient retries the same (client, req) on failover without re‑executing.

Connect applications with ClusterClient (§3). It rotates the address list and retries on Unavailable until it reaches the primary.

7b. Sharded deployment & cross-shard transactions

For horizontal scale, run K independent shard groups (each a cluster as above) plus one small sequencer group, with a router in front (kesseldb_server::router):

#![allow(unused)]
fn main() {
use kesseldb_server::router::{Router, serve_router, recover};

let router = std::sync::Arc::new(
    Router::new(vec![
        vec!["shard0a:7878".into(), "shard0b:7878".into(), "shard0c:7878".into()],
        vec!["shard1a:7878".into(), "shard1b:7878".into(), "shard1c:7878".into()],
    ])
    .with_sequencer(vec!["seqa:7878".into(), "seqb:7878".into(), "seqc:7878".into()]),
);
serve_router(listener, router.clone());      // speaks the ordinary client wire
// after a router restart, finish any in-flight cross-shard txns:
recover(&router).unwrap();
}

The router sends each request to the shard that owns its key (rendezvous mapping); schema/DDL is broadcast to every shard. A single‑shard transaction stays on its shard's own VSR group. A cross‑shard Op::Txn is decomposed into per‑shard slices, durably totally‑ordered by the sequencer, then applied by a deterministic decide → commit: it is atomic (a slice that would fail aborts the whole transaction on every shard), exactly‑once under client retry (use session‑framed clients for true exactly‑once), and recoverable (recover() re‑drives the ordered log idempotently after a router restart). This is deterministic (Calvin‑style), not blocking 2PC. Cross‑shard transactions are point‑op batches (Create/Update/Delete); SQL‑text routing is a separate later concern.

Cross‑shard reads (SP‑A). Op::Select / Op::QueryRows / Op::SelectFields / Op::SelectSorted automatically scatter to every shard and merge at the router. Clients send the same Op they would send to a K=1 deployment — the router does the fan‑out, the wire contract stays unchanged. When you scale shard count: parallel scatter latency is ≈ max(per‑shard scan latency) + merge overhead, so adding more shards keeps per‑query latency roughly flat while throughput scales linearly with K. SelectSorted produces byte‑identical output to a K=1 deployment for the same dataset (K‑invariance is locked across K ∈ {1, 2, 4, 8, 16} by a 425‑run property sweep at the merge layer + a real‑socket K=1↔K=4 byte‑ identical integration test). LIMIT cancellation propagates a shared cancel flag the instant the output buffer fills, so late shards don't keep the router pinned. V1 limitations (each a later spec): cross‑shard Aggregate / GroupAggregate reject with a clear error (SP‑B / SP‑D); SQL‑text routing for queries that COULD route to one shard by a key‑equality WHERE still fans out (SP‑E); FindBy / FindByComposite still route via per‑shard secondary indexes (extension to scatter is a follow‑up); sort‑key tie‑break is by (value, shard_id) not (value, object_id) (documented edge); a scatter read sees per‑shard snapshots taken at request‑arrival, NOT a cross‑shard consistent snapshot. The default failure mode is hard‑ fail (a single unavailable shard surfaces a clean error to the caller, never a silently partial result); a ScatterContext opt‑in for partial‑on‑timeout best‑effort mode exists at the scatter_and_merge_ctx API level for embeddable use.

7c. External sources (JSON/CSV over HTTP)

An external source is a named table whose rows are populated by fetching a remote JSON or CSV endpoint and materializing the result into a normal KesselDB type. Once materialized, the rows are queried with ordinary SQL — indexes, aggregates, joins, and constraints all apply.

Requires the external-sources cargo feature. Build and run the server with --features external-sources:

cargo build --release --features external-sources
cargo run --release --bin kesseldb --features external-sources -- 127.0.0.1:7878 ./data

Register a source

CREATE EXTERNAL SOURCE prices (
    ticker  CHAR(8)   NOT NULL FROM 'symbol',
    price   I64       NOT NULL FROM 'quote.last',
    volume  U64       NOT NULL FROM 'vol'
) FROM 'http://data.example.com/quotes.json'
  FORMAT JSON
  KEY ticker
  AUTH BEARER ENV 'PRICES_API_TOKEN'

• FROM '<path>' after a column name is the JSON dotted path to the value in each array element (FORMAT JSON) or the CSV header name (FORMAT CSV).
• FROM '<url>' after the column list is the HTTP endpoint.
• FORMAT JSON — expects a top-level JSON array of objects. FORMAT CSV — expects RFC 4180 with a header row.
• KEY <col> — the column whose value is the stable row identity. The same upstream key always maps to the same row; REFRESH upserts (create-if-absent / update-if-changed) without duplicating rows.
• AUTH BEARER ENV 'VAR' — send Authorization: Bearer $VAR where $VAR is read from the server's environment. No auth: omit the clause. Custom header: AUTH HEADER 'X-Api-Key' ENV 'VAR'.

Populate / refresh

REFRESH prices

REFRESH fetches the URL once, parses and type-checks every row, and submits a single atomic upsert batch through the replicated log. If any row fails type coercion the entire refresh is rejected and the prior data is unchanged. Re-REFRESH with the same upstream data is idempotent (same rows → same ids → same state, digest unchanged).

Query

SELECT * FROM prices WHERE ticker = 'AAPL'
SELECT ticker, price FROM prices ORDER BY price DESC LIMIT 10
SELECT COUNT(*) FROM prices WHERE volume > 1000000

prices is an ordinary KesselDB table — all SQL, indexes, aggregates, and joins work normally.

Remove a source

DROP EXTERNAL SOURCE prices

Removes both the materialized rows and the registered source definition.

Security — secret handling

Only the env-var name ('PRICES_API_TOKEN') is stored in the catalog and replicated. The actual secret value is read from the router's process environment at REFRESH time and never appears in any operation, WAL entry, or log line.

Export the secret in the environment of the server process before starting it:

export PRICES_API_TOKEN=sk-...
cargo run --release --bin kesseldb --features external-sources -- 127.0.0.1:7878 ./data

Honest boundaries

• Snapshot since last REFRESH. A source reflects only its last successful REFRESH; queries read the materialized snapshot, never live upstream. Re-run REFRESH whenever you need fresh data.
• HTTP and HTTPS. The fetch client speaks plain HTTP/1.1 for http:// sources. https:// is supported when the server is built with --features external-sources-tls (rustls client, bundled Mozilla webpki-roots, full certificate + hostname verification, no bypass under any flag). The TLS-terminating sidecar is now optional.
• Upsert only. Rows deleted from the upstream source are NOT automatically removed; only creates and updates are applied.

7d. Paginated & NDJSON sources

External sources support two additional capabilities (requires the external-sources feature, same as §7c):

• FORMAT NDJSON — one JSON object per line; otherwise identical to FORMAT JSON.
• Multi-page PAGE clause — a single REFRESH walks multiple pages and materializes the union. Three cursor forms are supported.

NDJSON one-liner

CREATE EXTERNAL SOURCE events (
    id     U64    NOT NULL FROM 'id',
    kind   BYTES  NOT NULL FROM 'type'
) FROM 'http://ingest.example.com/events.ndjson'
  FORMAT NDJSON
  KEY id

Paginated JSON with a next-URL in the response body

CREATE EXTERNAL SOURCE products (
    sku    BYTES  NOT NULL FROM 'sku',
    price  I64    NOT NULL FROM 'price_cents'
) FROM 'http://catalog.example.com/api/products'
  FORMAT JSON
  KEY sku
  ROWS 'data.items'
  PAGE NEXT JSON 'paging.next'

ROWS 'data.items' — the row array lives at that dotted path inside the envelope object (required when FORMAT JSON is combined with a body-cursor PAGE clause).

PAGE NEXT JSON 'paging.next' — after each page, extract the absolute next-page URL from paging.next in the envelope; stop when the field is absent, null, or an empty string.

Cursor form — opaque token

CREATE EXTERNAL SOURCE orders (
    order_id  U64   NOT NULL FROM 'id',
    total     I64   NOT NULL FROM 'total'
) FROM 'http://shop.example.com/api/orders'
  FORMAT JSON
  KEY order_id
  ROWS 'results'
  PAGE CURSOR JSON 'meta.cursor' PARAM 'cursor'

PAGE CURSOR JSON 'meta.cursor' PARAM 'cursor' — extract the opaque token from meta.cursor in the envelope; the next request is the original recipe URL with ?cursor=<token> appended (replacing any pre-existing cursor query parameter).

PAGE NEXT LINK — HTTP Link header

CREATE EXTERNAL SOURCE issues (
    id     U64    NOT NULL FROM 'id',
    title  BYTES  NOT NULL FROM 'title'
) FROM 'http://api.example.com/repos/acme/issues'
  FORMAT JSON
  KEY id
  PAGE NEXT LINK

Valid with any format (FORMAT JSON, FORMAT NDJSON, FORMAT CSV). Uses the Link: <url>; rel="next" response header as the next-page URL.

Compatibility rules (enforced at CREATE)

Bounded fetch — safety caps

All multi-page fetches are hard-bounded:

• MAX_PAGES = 1000 — a REFRESH walks at most 1,000 pages.
• MAX_TOTAL_BODY = 8 × 64 MiB — aggregate decompressed response bytes across all pages.
• Per-page body cap (64 MiB) still applies to each individual response.
• Loop detection — if the extracted next-URL or cursor token exactly equals one already seen in the current walk, REFRESH returns an error.

If any of these caps is exceeded, or if any page returns an HTTP error, parse error, or type-coercion failure, the entire REFRESH is aborted and nothing is materialized — prior data remains intact (all-or-nothing, same as a single-page refresh).

Honest boundaries (same as §7c, unchanged)

• Snapshot since last REFRESH. A source reflects only its last successful REFRESH; queries read the materialized snapshot, never live upstream.
• HTTP and HTTPS. Plain HTTP/1.1 for http:// sources. https:// when built with --features external-sources-tls (bundled Mozilla roots, full certificate + hostname verification, no bypass). Sidecar now optional.
• Upsert only. Rows deleted from the upstream source are not automatically removed.

7e. Object-store sources (S3 / Azure Blob)

An external source can read its bytes directly from an S3-compatible or Azure Blob object store — CREATE EXTERNAL SOURCE … FROM 's3://…' | 'az://…' — using the same fetch → decode → atomic-upsert pipeline as §7c and §7d. The difference is transport: the router builds a signed HTTPS GET (AWS SigV4 for S3; Azure Shared Key for Azure Blob), fetches the object body, and feeds it through the existing decoder. Pagination (PAGE …) is not applicable to object-store sources; a single object is fetched per REFRESH.

Requires the external-sources-objstore cargo feature (which implies external-sources-tls; the default build and plain --features external-sources remain http(s)://-only and pull no rustls/webpki/objstore):

cargo build --release --features external-sources-objstore
cargo run  --release --bin kesseldb --features external-sources-objstore -- 127.0.0.1:7878 ./data

S3 / S3-compatible (MinIO, Cloudflare R2, Ceph)

-- AWS S3 — IAM-key auth, inferred path-style URL from region + bucket/key
CREATE EXTERNAL SOURCE prices (
    ticker  BYTES  NOT NULL FROM 'symbol',
    price   I64    NOT NULL FROM 'quote.last'
) FROM 's3://my-bucket/data/prices.json'
  FORMAT JSON
  KEY ticker
  REGION 'us-east-1'
  AUTH OBJSTORE S3 KEYID ENV 'AWS_ACCESS_KEY_ID' SECRET ENV 'AWS_SECRET_ACCESS_KEY'

-- S3-compatible (MinIO / R2 / Ceph) — ENDPOINT overrides the host; REGION optional
CREATE EXTERNAL SOURCE events (
    id    U64   NOT NULL FROM 'id',
    kind  BYTES NOT NULL FROM 'type'
) FROM 's3://warehouse/events.ndjson'
  FORMAT NDJSON
  KEY id
  ENDPOINT 'https://minio.internal:9000'
  AUTH OBJSTORE S3 KEYID ENV 'MINIO_KEY' SECRET ENV 'MINIO_SECRET'

Clause rules for s3://:

• REGION '<r>' — required for AWS S3 unless ENDPOINT is supplied. Ignored for presigning purposes when ENDPOINT is given.
• ENDPOINT '<https-url>' — overrides the request host for path-style access (MinIO / R2 / Ceph / any S3-compatible). The value must start with https:// (rejected at CREATE if not).
• AUTH OBJSTORE S3 KEYID ENV '<idvar>' SECRET ENV '<secretvar>' — env-var names only; the actual key and secret are resolved from the router's process environment at each REFRESH and never appear in any op, WAL entry, or log line.

Azure Blob Storage

-- Azure Blob — ACCOUNT declared explicitly in the AUTH clause (or use ENDPOINT instead; exactly one)
CREATE EXTERNAL SOURCE catalog (
    sku    BYTES  NOT NULL FROM 'sku',
    price  I64    NOT NULL FROM 'price_cents'
) FROM 'az://my-container/catalog.json'
  FORMAT JSON
  KEY sku
  AUTH OBJSTORE AZURE ACCOUNT 'mystorageaccount' KEY ENV 'AZURE_STORAGE_KEY'

-- Custom / sovereign endpoint — ENDPOINT replaces the default host
CREATE EXTERNAL SOURCE archive (
    id     U64   NOT NULL FROM 'id',
    label  BYTES NOT NULL FROM 'label'
) FROM 'az://archive-container/records.ndjson'
  FORMAT NDJSON
  KEY id
  ENDPOINT 'https://mystorageaccount.blob.core.windows.net'
  AUTH OBJSTORE AZURE KEY ENV 'AZURE_STORAGE_KEY'

Clause rules for az://:

• ACCOUNT '<a>' — the Azure storage account name. Exactly one of ACCOUNT or ENDPOINT is required; both present is also accepted when the ENDPOINT is the account's canonical blob URL.
• ENDPOINT '<https-url>' — overrides the default https://<account>.blob.core.windows.net host. Must start with https:// (rejected at CREATE if not).
• AUTH OBJSTORE AZURE [ACCOUNT '<a>'] KEY ENV '<keyvar>' — the storage account shared key is resolved from the named env var at REFRESH time; never persisted.

Refresh & query

REFRESH prices     -- fetches, decodes, upserts; prior data intact on any error
SELECT * FROM prices WHERE ticker = 'AAPL'
DROP EXTERNAL SOURCE prices

REFRESH is all-or-nothing: any fetch error, HTTP error status, parse failure, or type-coercion failure aborts the entire operation and leaves the prior materialized rows unchanged. Re-REFRESH with the same upstream data is idempotent (same rows → same IDs → same digest, same as §7c).

Security

Signing. AWS SigV4 (HMAC-SHA256, no external crypto library — the same zero-dep SHA-256/HMAC-SHA256 already in the kessel-sm kernel); Azure Shared Key (HMAC-SHA256). Both are implemented entirely inside the kessel-objstore crate.

HTTPS-only, no bypass. Every object-store request goes over TLS (rustls + bundled Mozilla webpki roots, full certificate + hostname verification). There is no flag, env var, or clause to disable certificate verification.

Injection-safe. Container names, blob keys, and the S3 bucket/key are RFC-3986 percent-encoded before being placed in the request URI and the signing string. CRLF and query-parameter injection are blocked.

Secret references only. Only the env-var name strings are stored in the catalog trailer and replicated in the WAL. The actual key/secret values are resolved from the router's environment at each REFRESH, are never logged, never included in any operation or WAL entry, never appear in digest output, and are never surfaced in error messages.

Honest boundaries

• FORMAT PARQUET is supported for s3:// / az:// sources with the --features external-sources-objstore build (OBJ-2a, §7f below). See §7f for the precise scope (PLAIN/UNCOMPRESSED/GZIP/flat REQUIRED or OPTIONAL/V1 and V2 pages) and the supported-vs-deferred matrix.
• Iceberg manifests, prefix/multi-object listing, and STS/SAS/IMDS credential providers are explicit follow-ons (OBJ-3 through OBJ-5) and are rejected at CREATE with a clear error message.
• Single object per source. A REFRESH fetches exactly one object. Listing a prefix or walking a multi-object partition is OBJ-4.
• Upsert only. Same as §7c — rows deleted from the upstream object are not automatically pruned from the materialized table.
• Snapshot since last REFRESH. Queries read the last materialized snapshot; live object-store reads are never issued by a SELECT.

7f. FORMAT PARQUET for object-store sources

Current capability (SP125‑SP154, OBJ‑2c‑2 codec arc CLOSED at 6/7 codecs): FORMAT PARQUET reads real pyarrow 24.0.0 Parquet files end‑to‑end across the **flat REQUIRED + OPTIONAL × UNCOMPRESSED + Snappy + GZIP + zstd

• LZ4_RAW + Brotli × PLAIN + dictionary × V1 + V2 data pages × INT32 + INT64 + INT96 + FLBA + BYTE_ARRAY + DECIMAL (precision ≤ 38)** matrix. Vanilla pq.write_table(df) works zero‑flags for everything in that matrix; pyarrow output for every supported codec decodes for all tested fixtures including a 2000‑row zstd stress fixture exercising FseCompressed mode for all three LL/OF/ML codes simultaneously and pyarrow compression='brotli' round-trips via the SP154 zero-dep RFC 7932 decoder. Still typed‑Unsupported: legacy LZ4 framing (codec id 5; modern LZ4_RAW codec id 7 IS supported), 4+ deep nested groups (would be SP147), DECIMAL precision > 38, per‑page > 256 MiB (SP151 raised the historical 64 MiB cap to a 256 MiB default + added kessel_parquet::extract_with_cap for operators with known-trusted producers or memory-constrained ingest). All Parquet nested types supported (LIST, MAP, struct + arbitrary nesting up to 3-deep — OBJ-2c-5 fully closed at SP146).

The slice‑by‑slice history below records how the capability grew — kept verbatim for traceability — but the matrix above is the authoritative current scope.

OBJ-2b in progress: the RLE/bit-packing-hybrid primitive is implemented (SP102) but not yet wired. Until OBJ-2b-2/3/4 ship, FORMAT PARQUET still requires PLAIN-encoded, UNCOMPRESSED, REQUIRED columns (pyarrow use_dictionary=False, compression=None).

OBJ-2b-2 (SP103): dictionary-encoded Parquet (pyarrow default use_dictionary=True) is now supported for flat REQUIRED, UNCOMPRESSED, V1 files. Compression still requires compression=None (Snappy → OBJ-2b-3); nullable/OPTIONAL columns still unsupported (→ OBJ-2b-4).

OBJ-2b-3 (SP104): Snappy-compressed Parquet (pyarrow default compression='snappy') is now supported for flat REQUIRED, V1 files (dictionary or PLAIN). nullable/OPTIONAL columns still unsupported (→ OBJ-2b-4); gzip/zstd and Snappy pages >64 MiB → OBJ-2c.

OBJ-2b-4 (SP105): vanilla pq.write_table(df) — flat REQUIRED or OPTIONAL columns, UNCOMPRESSED or Snappy, PLAIN or dictionary, V1 — is now fully supported, including NULLs (OPTIONAL def-level 0 → PqValue::Null). The OBJ-2b arc is COMPLETE. REPEATED columns / repetition levels, nested/optional groups, gzip/zstd/lz4/brotli, INT96/DECIMAL, V2 data pages, and Snappy pages >64 MiB remain Unsupported (→ OBJ-2c).

OBJ-2c-1 (SP106): GZIP-compressed Parquet (pyarrow compression='gzip') is now supported for flat REQUIRED or OPTIONAL columns, PLAIN or dictionary encoding, V1 pages. The pure zero-dep RFC 1952 + RFC 1951 inflater composes with dictionary and OPTIONAL/def-levels via the existing page_payload seam; no other code path changed. Pages decompressed to more than 64 MiB are rejected (typed Unsupported). ZSTD/lz4/brotli, INT96/DECIMAL, V2 data pages, REPEATED/nested, and GZIP pages >64 MiB remain Unsupported (→ OBJ-2c-2+).

OBJ-2c-3 (SP107): DATA_PAGE_V2 data pages (pyarrow data_page_version='2.0') are now supported for the existing flat REQUIRED or OPTIONAL × UNCOMPRESSED|Snappy|GZIP × PLAIN|dict matrix. The V2 raw-level-split path reads the uncompressed def/rep level bytes directly, then decompresses only the value section; the shared scatter_nulls helper keeps the V1 OPTIONAL path byte-identical. OBJ-2c-2 (zstd) was resequenced/deferred to prioritise broader pyarrow compatibility. ZSTD/lz4/brotli, INT96/DECIMAL, REPEATED/nested (incl. V2 repetition levels), and pages >64 MiB remain Unsupported (→ OBJ-2c-2/4/5).

OBJ-2c-4 (SP108): INT96 timestamps and DECIMAL logical-type values are now decoded for the existing flat REQUIRED or OPTIONAL × UNCOMPRESSED|Snappy|GZIP × V1|V2 × PLAIN|dict matrix. INT96 physical columns decode to PqValue::Timestamp(i64 ns) via checked Julian-day arithmetic (nanoseconds since Unix epoch). DECIMAL logical-type columns decode to PqValue::Decimal { unscaled: i128, scale: i32 } for physical types INT32, INT64, and FixedLenByteArray (BYTE_ARRAY DECIMAL is covered by hand-KATs only; pyarrow 24.0.0 does not write it). FLBA non-DECIMAL columns (e.g., FLBA-UUID) decode to PqValue::Bytes. Today, pq_to_cell maps Timestamp → Cell::Text (Unix-ns string) and Decimal → Cell::Text (unscaled-integer string); mapping via FieldKind::I64 or FieldKind::I128 (unscaled) works end-to-end. Coercion to FieldKind::Timestamp (for Timestamp) and FieldKind::Fixed{scale} (for Decimal) are immediate follow-up items. DECIMAL precision must be 1..=38 (backed by i128); precision > 38 is rejected with Unsupported. ZSTD/lz4/brotli, REPEATED/nested (incl. V2 rep-levels), and pages >64 MiB remain Unsupported (→ OBJ-2c-2/5).

OBJ-2c-4 follow-up (SP151): the historical 64 MiB per-page cap is lifted to 256 MiB default + a configurable operator knob. Pyarrow writers emit pages above 64 MiB on common shapes (high-cardinality dictionary pages, large value pages on many-row row groups); pre-SP151 those tripped a typed Unsupported with the 64 MiB cap value. Post-SP151:

• kessel_parquet::extract(bytes, wanted) uses DEFAULT_MAX_PAGE_SIZE = 256 * 1024 * 1024 — covers every pyarrow shape seen in the wild without operator intervention.
• kessel_parquet::extract_with_cap(bytes, wanted, max_page_size) is the operator knob. Raise above 256 MiB up to the per-codec module ceiling (also 256 MiB) for known-trusted producers; lower for memory-constrained ingest; cap=0 is the kill-switch that rejects every page (useful when sanitising hostile input). The cap is enforced as a thread-local set on entry and restored on return (RAII, including panic-unwind).
• Pages above the cap return Unsupported naming SP151, the extract_with_cap operator knob, and the cap value so an operator hitting the cap in production has a direct path to raise it.
• Defense in depth: the four per-codec module ceilings (SNAPPY_MAX_DECOMP, GZIP_MAX_DECOMP, ZSTD_MAX_DECOMP, LZ4_MAX_DECOMP) all bumped from 64 MiB → 256 MiB in lockstep. Even a caller passing usize::MAX to extract_with_cap can't OOM the decoder — the per-codec ceiling still gates allocation.

FORMAT PARQUET is supported for s3:// and az:// sources when the server is built with --features external-sources-objstore. Plain http:// / https:// URLs are rejected with a clear message if FORMAT PARQUET is specified — Parquet is object-store only. PAGE and ROWS clauses are also rejected at CREATE with FORMAT PARQUET (they are not applicable: a Parquet object is self-describing and multi-row-group; row selection is column-map driven, not page-cursor driven).

Requires --features external-sources-objstore (same as §7e); the default build and plain --features external-sources do not compile Parquet support and do not link any parquet/objstore/rustls dependency.

SQL syntax

CREATE EXTERNAL SOURCE readings (
    sensor_id  U64    NOT NULL FROM 'sensor_id',
    temp_c     I64    NOT NULL FROM 'temp_celsius',
    label      BYTES  NOT NULL FROM 'label'
) FROM 's3://my-bucket/data/readings.parquet'
  FORMAT PARQUET
  KEY sensor_id
  REGION 'us-east-1'
  AUTH OBJSTORE S3 KEYID ENV 'AWS_ACCESS_KEY_ID' SECRET ENV 'AWS_SECRET_ACCESS_KEY'

• FROM '<col_name>' after each column is the flat Parquet leaf column name (ColumnMap.source). It must be a leaf column present in the Parquet schema at the top level (no nested group path syntax in OBJ-2a).
• All other clauses (REGION, ENDPOINT, AUTH OBJSTORE S3/AZURE, KEY) are identical to §7e.
• REFRESH and DROP EXTERNAL SOURCE work identically to §7e.

Parquet scope: what is currently supported (OBJ-2a → OBJ-2c-5 SP146 — arc FULLY CLOSED)

SP143 nested decode: SP143 lifts the OBJ-2c flat-schema restriction for List<primitive> columns specifically. Each List<primitive> row's value is decoded as PqValue::List(Vec<PqValue>) per Dremel-style record assembly using the canonical 3-node LIST encoding pattern (outer group LIST { repeated middle group { primitive element }}). 5 pyarrow 24.0.0 fixtures roundtrip-tested (list_i64_required, list_i64_optional, list_string, optional_list_i64, list_with_null_items).

SP144 nested decode: SP144 lifts the OBJ-2c-5 nested rejection for canonical Map<K, V> and struct-of-primitives columns. Map values decode as PqValue::Map(Vec<(PqValue, PqValue)>) via assemble_map_kv over parallel key+value streams; struct values decode as PqValue::Struct(Vec<(String, PqValue)>) via assemble_struct zipping N field columns. Map keys MUST be REQUIRED per Parquet spec (rejected as Bad otherwise). 5 pyarrow 24.0.0 fixtures roundtrip-tested (map_string_i64, optional_map_string_i64, map_string_string, struct_i64_string, optional_struct).

SP145 deep nesting: SP145 lifts the 4 SP145-named rejections in classify_column_plan via per-shape composition (BOLD V1 — no full Dremel automaton needed for the shapes Parquet writers actually produce). 4 new ColumnKind variants + a recursive StructField.nested: Option<Box<ColumnKind>> enable: (a) List<List<T>> via assemble_list_of_list_primitive (max_rep_level=2 generalized assembler); (b) List<struct<...>> via assemble_list_of_struct (field-zip per item slot using the shared REPEATED-ancestor rep stream); (c) Map<K, struct<...>> via assemble_map_of_struct; (d) Map<K, List<T>> (BOLD cross-product) via assemble_map_of_list; (e) struct<List/Map/struct> via recursive classify_nested_group_child

• decode_field_by_kind dispatch. 7 pyarrow 24.0.0 fixtures roundtrip-tested.

SP146 deep-nesting follow-ups: SP146 closes the 3 cross-products SP145 V1 deferred (each named SP146 follow-up in the SP145-era source error messages). 3 new ColumnKind variants + 3 new assemblers + 1 new classify helper: (a) List<List<List<T>>> 3-deep (max_rep_level=3) via assemble_list_of_list_of_list_primitive (8-case classifier + 3-level stack outer/middle/inner accumulators); (b) List<Map<K, V>> via assemble_list_of_map_kv (outer-list-of-inner-maps driven off shared K/V rep stream); (c) Map<K1, Map<K2, V>> via assemble_map_of_map_kv (outer-map-of-inner-maps with outer K at max_rep=1 + inner K/V at max_rep=2). 3 pyarrow 24.0.0 fixtures roundtrip-tested (list_of_list_of_list_i64, list_of_map_string_i64, map_string_map_string_i64) — all GREEN on first try. OBJ-2c-5 arc FULLY CLOSED with NO follow-ups remaining — every nested Parquet shape pyarrow writes is now decodable.

What is NOT supported (rejected at REFRESH with a precise error)

The following trigger a typed PqError (surfaced as a REFRESH failure; prior materialized data is left intact — all-or-nothing, same as every other format):

• REPEATED columns / repetition levels outside the canonical LIST<primitive> (SP143), MAP<K, V> (SP144), List<List<T>> / List<struct> / Map<K, struct> / Map<K, List<T>> / struct<List/Map/struct> (SP145), List<List<List<T>>> / List<Map<K,V>> / Map<K1, Map<K2,V>> (SP146) shapes — rejected with Unsupported(...). All Parquet nested types up to 3-deep are now supported (OBJ-2c-5 arc fully closed).
• 4-layer-deep nesting (List<List<List<List<T>>>> etc.) — rejected with Unsupported("...: SP147 follow-up"). The per-shape composition pattern from SP145/SP146 generalizes to one more level the same way; no pyarrow corpus exercises this depth yet.
• Brotli compression (codec id 4) — fully supported (SP154). A hand-rolled zero-dep RFC 7932 Brotli decoder ships across 12 layers (bit reader → stream/metablock framing → simple+complex prefix codes → NBLTYPES + NPOSTFIX/NDIRECT + context-map headers → 704-symbol insert-and-copy command alphabet → 64-symbol distance prefix code + recent-distance ring → 122,784-byte static dictionary blob + 121 Appendix B transforms → compressed-metablock orchestration → flat output buffer with pre-stream-zero copy semantics), comparable in scope to the SP125-SP140 zstd arc. The decoder enforces V1 reductions matching the common pyarrow-emitted shape (NBLTYPES=1, NPOSTFIX=0+NDIRECT=0, NTREES=1 for both CMAPs, identity-only dictionary transforms); files that exceed those reductions surface typed BrotliMetablockError::{UnsupportedBlockTypes, UnsupportedDistanceParams, Context, Dictionary, ...} mapped to Unsupported with a named SP154-followup pointer. Pyarrow's compression='brotli' round-trips byte-identical for the standard flat-i64 + flat-BYTE_ARRAY shape (locked by the pyarrow_brotli_flat integration KAT). Closes OBJ-2c-2 codec matrix at 6/7 codecs supported (UNCOMPRESSED, Snappy, GZIP, Zstd, LZ4_RAW, Brotli; LZO remains deprecated, legacy LZ4 codec id 5 rejected with named pointer).
• Legacy LZ4 compression (codec id 5, deprecated Hadoop framing) — rejected with Unsupported("LZ4 (deprecated Hadoop framing) — use LZ4_RAW; SP149 follow-up if needed"). Pyarrow stopped writing this variant in v8; the modern LZ4_RAW (codec id 7) is fully supported.
• Pages above the per-call max_page_size cap — rejected with Unsupported("<page kind> size <N> exceeds max_page_size cap <cap>: SP151 (raise via kessel_parquet::extract_with_cap)"). The default cap is 256 MiB (4× the historical 64 MiB limit; SP151). The per-codec module ceilings (SNAPPY_MAX_DECOMP, GZIP_MAX_DECOMP, ZSTD_MAX_DECOMP, LZ4_MAX_DECOMP) are also 256 MiB and act as the absolute defense-in-depth ceiling — extract_with_cap can lower the cap but cannot raise it above the per-codec ceiling.
• DECIMAL precision > 38 — rejected with Unsupported("DECIMAL precision … (must be 1..=38): OBJ-2c-4"). DECIMAL backed by i128 (≤ 38 digits) is supported; wider types are not.
• Pre-1970 INT96 through FieldKind::Timestamp coerce — the decoder produces a correct negative-nanosecond PqValue::Timestamp; the FieldKind::Timestamp coerce path in pq_to_cell is typed FetchError::Type at coerce time for negative values. Map to FieldKind::I64 for any sign (unscaled Unix ns); immediate follow-up: signed-Timestamp FieldKind extension.
• DECIMAL → FieldKind::Fixed coerce — pq_to_cell Decimal arm is typed FetchError::Type at coerce time when the target column is FieldKind::Fixed (Fixed is internal-only today); immediate follow-up: to_field_bytes Fixed arm. Mapping DECIMAL → FieldKind::I128/I64 (unscaled integer) works today.
• BYTE_ARRAY DECIMAL via pyarrow — hand-KAT-only coverage; pyarrow 24.0.0 does not write BYTE_ARRAY DECIMAL (it always chooses INT32, INT64, or FLBA based on precision). The decode arm is implemented and KAT-tested; real-fixture coverage is deferred until a writer that emits it is available.
• A mapped column name absent from the Parquet schema — rejected with Bad("column \` not found in Parquet schema")`.

None of the above are decoded silently or partially. Failure is precise, typed, and fail-closed — the error message names the OBJ-2c follow-on that will address it.

Producing a compatible Parquet file

A file compatible with OBJ-2a can be written with pyarrow:

import pyarrow as pa, pyarrow.parquet as pq

schema = pa.schema([
    pa.field("sensor_id", pa.int64(), nullable=False),
    pa.field("temp_celsius", pa.int64(), nullable=False),
    pa.field("label", pa.large_binary(), nullable=False),
])
table = pa.table({
    "sensor_id":   pa.array([1, 2, 3], type=pa.int64()),
    "temp_celsius": pa.array([22, 18, 25], type=pa.int64()),
    "label":        pa.array([b"A", b"B", b"C"], type=pa.large_binary()),
})
pq.write_table(table, "readings.parquet",
               version='1.0',
               use_dictionary=False,
               compression="none",
               data_page_version="1.0")

Key options: use_dictionary=False (forces PLAIN encoding), compression="none" (UNCOMPRESSED), data_page_version="1.0" (V1 pages). Multi-row-group files are supported — all row groups are iterated in order. data_page_version="2.0" (DATA_PAGE_V2) is also supported as of OBJ-2c-3 (§7f) for the same flat REQUIRED|OPTIONAL × UNCOMPRESSED|Snappy|GZIP × PLAIN|dict matrix.

Physical-type-to-KesselDB-column mapping

The mapping goes through the same coerce::to_field_bytes path the JSON decoder uses, so the same logical value yields identical FieldKind bytes regardless of whether it arrived as JSON or Parquet.

8. Authentication, quotas & backpressure

Configured via ServerConfig and the *_cfg entry points:

#![allow(unused)]
fn main() {
use kesseldb_server::{run_cfg, ServerConfig};

run_cfg("0.0.0.0:7878", "./data", ServerConfig {
    token: Some(b"my-shared-secret".to_vec()), // None = open (default)
    max_conns: 1024,        // refuse connections past this
    max_inflight: 4096,     // shed load to `Unavailable` past this
})?;
}

• Auth: when token is set, the first frame on every connection must be the token; it is compared in constant time (no byte‑timing the secret). Clients use Client::connect_authed / ClusterClient::with_token.
• Connection quota: connections past max_conns are refused immediately.
• Backpressure: when max_inflight requests are queued, new ones get OpResult::Unavailable instead of growing the queue unbounded.

Transport encryption: KesselDB does not implement TLS in‑process (that would require bundling cryptography and break the zero‑dependency design). Run it behind a TLS‑terminating reverse proxy, or on a private/encrypted network (WireGuard, tailnet, VPC). The wire is plaintext but token‑authenticated. Or build with --features http-gateway,tls to terminate HTTPS in-process on ServerConfig.http_tls_addr — see §HTTP gateway below.

9. PostgreSQL clients (psql, pgcli, JDBC, psycopg, pgx, …)

KesselDB speaks the PostgreSQL Frontend/Backend Protocol v3.0 — the same wire libpq, psql, pgcli, JDBC, psycopg, pgx, tokio-postgres, sqlx-pg, Diesel-pg, GORM-pg, Drizzle-pg, Prisma-pg, … all speak. Built behind the opt-in pg-gateway feature flag so the default binary stays lean.

Both Simple Query AND Extended Query are supported (V1.1, 2026-05-29). SP-PG V1 shipped the Simple Query path (Q message) so psql and any client that does its own SQL formatting works. SP-PG-EXTQ V1 (2026-05-29) adds the full Extended Query message set (P / B / D / E / S / C / H), which is what every modern ORM (psycopg2/3, asyncpg, SQLAlchemy, Drizzle, Prisma, JDBC default, sqlx, pgx, Diesel) uses on connect — they probe via Parse + Bind + Sync before falling back to Simple Query. KesselDB now satisfies that probe end-to-end. A real psycopg2.connect(...) + cur.execute("SELECT * FROM pgtest WHERE id = %s", (42,)) returns real rows; broader ORM-suite smoke for SQLAlchemy/JDBC/ Drizzle/Prisma is covered in the compat matrix below — the wire surface IS lit.

Enable the PG listener

# Build (or download the release binary, which already includes pg-gateway):
cargo build --release -p kesseldb-server --features pg-gateway

# Start kesseldb with the PG listener bound:
KESSELDB_TOKEN=secret \
KESSELDB_PG_ADDR=127.0.0.1:5432 \
  ./target/release/kesseldb 127.0.0.1:7878 /tmp/kessel.db
# => KesselDB listening on 127.0.0.1:7878, data dir /tmp/kessel.db, pg=127.0.0.1:5432

Two env vars matter:

• KESSELDB_TOKEN — the operator's shared-secret Bearer token (the same one the HTTP gateway uses). The PG listener REQUIRES a token to be set; closed-mode-without-token rejects the connection with 28000 invalid_authorization_specification. The PG-wire SCRAM exchange uses this token as the SCRAM password input (one credential surface; rotating the token rotates HTTP-Bearer, WS, and PG-SCRAM atomically).
• KESSELDB_PG_ADDR — host:port to bind the PG listener on. Standard default is :5432; bind to 127.0.0.1:5432 for localhost-only, or 0.0.0.0:5432 to accept remote connections. PG and HTTP have independent connection caps so a misbehaving pgcli cannot starve HTTP clients.

When all three listeners are active (binary + HTTP + PG), the startup line surfaces every bound address:

KesselDB listening on 127.0.0.1:7878, data dir ./data, http=127.0.0.1:8080, pg=127.0.0.1:5432

Connect with psql

PGPASSWORD=$KESSELDB_TOKEN psql -h localhost -p 5432 -U test "SELECT 1"

The -U test username can be anything — V1 is multi-user-deferred (the SCRAM exchange authenticates against the Bearer token regardless of the PG user field). Interactive sessions work too:

PGPASSWORD=$KESSELDB_TOKEN psql -h localhost -p 5432 -U test
kessel=> CREATE TABLE users (id i64 PK, name char(64));
CREATE TABLE
kessel=> INSERT INTO users (id, name) VALUES (1, 'Alice'), (2, 'Bob');
INSERT 0 2
kessel=> SELECT * FROM users;
 id |  name
----+-------
  1 | Alice
  2 | Bob
(2 rows)
kessel=> \q

Connect from JDBC

Standard org.postgresql:postgresql driver:

String url = "jdbc:postgresql://localhost:5432/kessel";
Properties props = new Properties();
props.setProperty("user", "test");
props.setProperty("password", System.getenv("KESSELDB_TOKEN"));
Connection conn = DriverManager.getConnection(url, props);
PreparedStatement stmt = conn.prepareStatement("SELECT * FROM users");
ResultSet rs = stmt.executeQuery();
while (rs.next()) {
    System.out.println(rs.getLong("id") + " " + rs.getString("name"));
}

Connect from Python (psycopg2/psycopg3)

After SP-PG-EXTQ V1 (2026-05-29) parameterized queries through %s placeholders work end-to-end — psycopg2 sends them via Extended Query (Parse / Bind / Execute / Sync) and KesselDB dispatches every frame through EngineApply::apply_sql after substituting the bind values.

import os
import psycopg2

conn = psycopg2.connect(
    host="localhost",
    port=5432,
    user="test",
    password=os.environ["KESSELDB_TOKEN"],
    dbname="kessel",
)
cur = conn.cursor()

# 1. Simple Query path (no placeholders) — works since SP-PG V1:
cur.execute("CREATE TABLE pgtest (id BIGINT, name CHAR(64))")
cur.execute("INSERT INTO pgtest (id, name) VALUES (42, 'Alice')")
cur.execute("SELECT * FROM pgtest")
print(cur.fetchall())           # → [(42, 'Alice')]

# 2. Extended Query path (parameterized) — works since SP-PG-EXTQ V1:
cur.execute("SELECT * FROM pgtest WHERE id = %s", (42,))
print(cur.fetchall())           # → [(42, 'Alice')] — real round-trip

Connect from SQLAlchemy

SQLAlchemy uses the psycopg2 or psycopg3 driver under the hood and probes via Extended Query on engine.connect(). With SP-PG-EXTQ V1 those probes return without 08P01 protocol_violation so engine.connect() succeeds:

import os
from sqlalchemy import create_engine, text

url = (
    f"postgresql+psycopg2://test:{os.environ['KESSELDB_TOKEN']}"
    f"@localhost:5432/kessel"
)
engine = create_engine(url)

with engine.connect() as conn:
    rows = conn.execute(
        text("SELECT * FROM pgtest WHERE id = :id"),
        {"id": 42},
    ).fetchall()
    print(rows)                 # → [(42, 'Alice')]

The ORM-layer scope (declarative models, autoflush, the full SQLAlchemy expression language) depends on which subset of catalog SQL SQLAlchemy emits — synthetic-peer KATs verify the connect + probe + simple parameterized SELECT shape.

SQLAlchemy ORM (declarative models) — full CRUD

A real SQLAlchemy 2.0 declarative-ORM CRUD workload (NOT text() / raw cursor.execute) works end-to-end against KesselDB. Closing three keystone ORM-shape gaps + two DDL-spelling gaps took the declarative-ORM smoke from 2/8 → 7/7 (full CRUD pass):

The model used: class User(Base): id = Column(BigInteger, primary_key=True); name = Column(String(32)). Definition: a qualified column (t.col) is accepted lenient (the qualifier is stripped) in projection / WHERE / SET / ORDER BY / GROUP BY; an explicit projection list is rendered at the PG-wire layer; col = ANY (ARRAY[…]) desugars to IN (…). A qualified query compiles to the BYTE-IDENTICAL Op as its bare equivalent (determinism preserved).

Residual follow-ups: SP-PG-SQL-UPDATE-WHERE-GENERAL (non-PK / multi-row UPDATE/DELETE WHERE), SP-PG-SQL-QUALIFIER-STRICT (strict qualifier validation), SP-PG-SQL-FROM-ALIAS, SP-PG-SQL-ANY-SUBQUERY, SP-PG-ORM-RELATIONSHIPS / SP-PG-ORM-ALEMBIC.

Autoincrement models (BIGSERIAL / INSERT … RETURNING) — 2026-06-03

Real ORM models overwhelmingly use autoincrement: the application declares id = Column(BigInteger, primary_key=True, autoincrement=True), NEVER supplies id, and the ORM reads the DB-assigned id back via INSERT … RETURNING id. SP-PG-SERIAL-RETURNING lights this up end-to-end:

CREATE TABLE widgets (id BIGSERIAL PRIMARY KEY, name VARCHAR(32));
INSERT INTO widgets (name) VALUES ('gadget') RETURNING id;   -- → 1
INSERT INTO widgets (name) VALUES ('sprocket') RETURNING id; -- → 2
INSERT INTO widgets (name) VALUES ('z') RETURNING id, name;  -- → (3, 'z')

• A BIGSERIAL/SERIAL/SMALLSERIAL column that is the PRIMARY KEY becomes a deterministic autoincrement: an INSERT that omits the id is assigned the next per-table sequence value by the engine. The id is the row's ObjectId (so by-PK SELECT/UPDATE/DELETE WHERE id = n address it) AND is stored in the id column (so SELECT id reads it).
• Determinism: the sequence counter lives in the replicated state digest and advances ONLY on the deterministic apply thread, in op-number order — every replica computes the identical gap-free sequence, and a crash + WAL replay resumes it exactly. No RNG, no wall-clock.
• RETURNING: INSERT … RETURNING col1, col2, … emits a result row with the requested columns (the assigned id and/or client-supplied values), on both the Simple- and Extended-Query paths.
• Multi-row RETURNING + RETURNING * (SP-PG-RETURNING-MULTIROW-STAR): a batched INSERT … VALUES (…),(…),(…) RETURNING id returns N DataRows (one assigned id per row, in insertion order), and RETURNING * expands to every table column. This closes the SQLAlchemy DEFAULT-config gap (see below).

Zero-config SQLAlchemy (2026-06-03). KesselDB now works with SQLAlchemy's OUT-OF-THE-BOX engine config — create_engine(url) with NO use_insertmanyvalues=False. SQLAlchemy 2.0's DEFAULT use_insertmanyvalues=True BATCHES a flush of multiple pending objects into ONE statement (its insertmanyvalues form: INSERT … SELECT … FROM (VALUES …) AS sen(…) ORDER BY sen_counter RETURNING …) and expects N rows back. The gateway desugars that form to the plain multi-row INSERT … VALUES (…),(…) RETURNING … the engine handles, so a batched session.add_all([a,b,c]); session.commit() reads back every DB-assigned id. SQLAlchemy DEFAULT-config CRUD: 5/5.

The single-row autoincrement path is 6/6.

V1 out-of-scope (named follow-ups): SP-PG-SQL-RETURNING-DML (UPDATE/DELETE RETURNING), SP-PG-SEQUENCE-DDL (CREATE SEQUENCE / nextval/setval), SP-PG-SERIAL-NONPK (a SERIAL column that is not the PK), SP-PG-RETURNING-EXPR (RETURNING id + 1 / expressions).

SQLAlchemy relationships / JOINs (multi-table FK) — 2026-06-03

SP-PG-ORM-RELATIONSHIPS validated a real SQLAlchemy 2.0 two-model FK-relationship workload (Author 1—N Book, declarative relationship() + ForeignKey) — the relational core. 4/4 stages PASS:

The two keystone fixes: (A) kessel-sql now accept-and-skips FK DDL (table-constraint + inline REFERENCES, incl. ON DELETE/UPDATE actions); (B) the PG-wire gateway renders an inner-equi-JOIN result — it decodes the engine's embedded combined schema, recovers the projection from the SQL (kessel_sql::join_projection), and emits the projected columns. Before this arc a JOIN hit the "only renders SELECT *" error even though the engine joined. V1 out-of-scope (named follow-ups): SP-PG-DDL-FK-ENFORCE (DONE 2026-06-03 — DDL FK is now ENFORCED end-to-end, bad child INSERT → 23503), SP-PG-SQL-OUTER-JOIN (LEFT — DONE) + SP-PG-SQL-RIGHT-FULL-JOIN (RIGHT/FULL — DONE 2026-06-03, the full INNER/LEFT/RIGHT/FULL matrix on a binary join), SP-PG-SQL-JOIN-ALIAS, SP-PG-SQL-JOIN-AGG. (SP-PG-SQL-JOIN-WHERE — filtered joins — shipped, see below; SP-PG-SQL-MULTI-JOIN — chained 3+ table INNER joins — SHIPPED 2026-06-03.) Transcript: docs/superpowers/sppgormrelationships-smoke-2026-06-03.txt.

Real multi-model app (blog) — capstone

The truest real-world-readiness test: a realistic three-model SQLAlchemy 2.0 blog application (User 1—N Post 1—N Comment, FKs + declarative relationship() with back_populates, insertmanyvalues batching ON — the default) exercising the full query range a real app uses, back-to-back. 8/8 stages PASS, every query returning REAL data:

Two surgical correctness fixes closed the only two gaps the workload surfaced: (1) the kessel-sql lexer now handles the SQL-standard doubled-quote string escape 'bob''s post' → bob's post (the previous lexer truncated at the first inner ' — this would break ANY app with an apostrophe in its data); (2) the gateway now renders a projection-list SELECT with ORDER BY (which lowers to Op::SelectSorted, returning FULL records, the projection dropped at the engine layer) by decoding the full records and re-projecting the requested columns (with proper NULL fidelity via the record's null bitmap). Neither touches the engine apply path or Op wire encoding; seed-7 + 3-replica determinism holds. This is the headline statement: a realistic multi-model app composes end-to-end, 8/8. Transcript: docs/superpowers/sppgormrealapp-smoke-2026-06-03.txt.

Filtered joins (JOIN … WHERE) — SP-PG-SQL-JOIN-WHERE, 2026-06-03

SELECT a.name, b.title FROM a JOIN b ON a.id = b.a_id WHERE b.title = $1 [AND a.name = $2] — the most common real-app join beyond a bare join (SQLAlchemy query.join(Book).filter(Book.title == x)). Op::Join gained an optional combined-schema filter program: the engine joins, then runs a kessel-expr predicate per combined row, keeping only matches. kessel-sql compiles the qualified WHERE after the ON clause against the combined (a-fields ++ b-fields) schema — a.x resolves to the left field, b.y to the right; a bare col resolves by suffix with an ambiguity error when it exists in both tables. AND/OR/NOT/IN/BETWEEN/LIKE and params all work over the join (the full WHERE grammar runs against the combined type). The gateway render is reused unchanged (a filtered join just returns fewer combined rows). The wire change is additive (the filter is a trailing optional field — a bare join is byte-identical to the pre-arc frame), and the filter is a pure function of the combined row, so seed-7 + 3-replica determinism holds. V1 out-of-scope (named follow-ups): SP-PG-SQL-JOIN-ORDERBY (JOIN … WHERE … ORDER BY/LIMIT over the combined schema), plus the inherited OUTER / MULTI / ALIAS / AGG follow-ups.

Plain (single-table) GROUP BY render — SP-PG-SQL-PLAIN-GROUP-RENDER, 2026-06-03

SELECT category, COUNT(*) [AS n] [, SUM(price), AVG(price), MIN(price), MAX(price)] FROM products GROUP BY category [HAVING …] [ORDER BY …] [LIMIT …] — the everyday analytics / ORM aggregation. The planner + state machine already compiled and executed plain GROUP BY (Op::GroupAggregate / Op::GroupAggregateMulti) and HAVING already filtered at the SM layer, but the PG-wire gateway had no render branch: render_select_got only routed group-aggregates through render_join_group_aggregate, which REQUIRES a JOIN, so a plain SELECT g, COUNT(*) … GROUP BY g fell through to the bottom render error (0A000 only renders SELECT *) even though the engine grouped correctly. This arc adds kessel_sql::plain_group_aggregate (recovers the output column shape: group key + each aggregate's out-name + source column) and render_plain_group_aggregate (decodes the value-only group stream, types the group key from the FROM-table schema, types aggregate OIDs per kind: COUNT/SUM → int8, AVG → numeric, MIN/MAX → source-column type). Render-only — no Op or wire-format change, so corpus / partition / 3-replica byte-identity is untouched. Update (SP-PG-SQL-GROUP-SORT-LIMIT): a trailing ORDER BY … LIMIT … OFFSET … on a plain group-agg is now engine-applied — the group ops carry an additive, marker-guarded GroupSort (sort by a projected aggregate or the group key; LIMIT/OFFSET after the sort; HAVING filters first), so top-N-per-group analytics works and a no-ORDER-BY frame stays byte-identical. Smokes: scripts/sppgsqlplaingrouprender-smoke.py (render), scripts/sppgsqlgroupsortlimit-smoke.py (sort/limit/offset).

Django ORM (the other dominant Python ORM) — 2026-06-03

A real Django 6.0 ORM workload (models + schema_editor DDL + ORM CRUD, via psycopg3) runs against KesselDB to prove ORM breadth beyond SQLAlchemy. After quoted-identifier support (the keystone — kessel-sql's lexer now accepts SQL-standard double-quoted delimited identifiers), the smoke advanced from 2/8 → 6/8. The unexpected char '"' boundary is fully gone; every genuine ORM CRUD statement now executes.

Quoted identifiers are fully accepted. kessel-sql's lexer accepts "ident" as a delimited identifier (case-preserving, "" escape) everywhere a bare identifier works — table, column, qualifier, in DDL / DML / projection / WHERE / SET / RETURNING. Quoted idents lower to the SAME token as the bare spelling, so Django's quoted DDL and DML round-trip on the same catalog names. No regression: the SQLAlchemy ORM smoke stays 7/7 (it emits unquoted identifiers).

Residual gaps are the two already-named follow-ups, not quoting:

• SP-PG-DDL-IDENTITY (P1) — Django 6's default BigAutoField renders … GENERATED BY DEFAULT AS IDENTITY, not BIGSERIAL; the create_model DDL still needs this spelling accepted as a BIGSERIAL-autoincrement alias.
• SP-PG-SQL-AGG-ALIAS-RENDER (P2) — SELECT COUNT(*) AS "__count" FROM t (Django .count()/.aggregate()): parse COUNT(*) AS <alias> + PG-wire render of a FROM-ful aggregate projection. (Probed directly: the quoted DELETE executes; only the trailing count trips.)

Lower-priority follow-ups remain SP-PG-DJANGO-INTROSPECT (P3 — connection.introspection for manage.py migrate/inspectdb) and SP-PG-SAVEPOINT (P3 — nested transaction.atomic() savepoints). Transcripts: docs/superpowers/sppgormdjango-smoke-2026-06-03.txt (boundary characterization, 2/8) and docs/superpowers/sppgsqlquotedident-django-smoke-2026-06-03.txt (post-quoted-ident, 6/8).

Supported GUI / admin tools

After SP-PG-CAT (V1 of the pg_catalog stubs arc), GUI admin / BI tools that issue catalog-introspection queries on connect now see synthesized responses instead of 42P01 undefined_table. The following tools have been verified via synthetic-peer KATs driving each tool's verbatim connect / introspection SQL through the catalog hook:

Sample interactive session through psql:

$ PGPASSWORD=$KESSELDB_TOKEN psql -h localhost -p 5432 -U test kessel
psql (14.10, server PostgreSQL 14.0 (KesselDB 1.0))
kessel=> CREATE TABLE users (id I64 NOT NULL, email CHAR(64) NOT NULL);
CREATE TABLE
kessel=> CREATE UNIQUE INDEX ON users (email);
CREATE INDEX
kessel=> \dt
        List of relations
 Schema | Name  | Type  | Owner
--------+-------+-------+----------
 public | users | table | kesseldb
(1 row)

kessel=> \d users
                Table "public.users"
 Column | Type | Collation | Nullable | Default
--------+------+-----------+----------+---------
 id     | int8 |           | not null |
 email  | text |           | not null |
Indexes:
    "users_email_idx" UNIQUE, btree (email)

kessel=> SELECT version();
                   version
---------------------------------------------
 PostgreSQL 14.0 (KesselDB 1.0)
(1 row)

kessel=> SELECT * FROM information_schema.tables
         WHERE table_schema NOT IN ('pg_catalog', 'information_schema');
 table_catalog | table_schema | table_name | table_type
---------------+--------------+------------+------------
 kesseldb      | public       | users      | BASE TABLE
(1 row)

Real psql session (verified 2026-05-28)

Captured from a real psql 16.14 libpq client driving the kesseldb-server binary (built with --features pg-gateway,http-gateway) on a Linux reference server. The server was started with:

KESSELDB_TOKEN=admin KESSELDB_PG_ADDR=127.0.0.1:5532 \
  ./target/release/kesseldb 127.0.0.1:6532 /tmp/kdb-data
# => KesselDB listening on 127.0.0.1:6532, data dir /tmp/kdb-data, pg=127.0.0.1:5532

Every command and its actual response are shown below. The session exercises authentication, the version() helper, \dt empty + populated, CREATE TABLE with the canonical PG BIGINT type (NOT KesselDB's I64 spelling), INSERT (single + multi-row), SELECT *, \d <table>, and \dn schema-list.

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c "SELECT version();"
            version
--------------------------------
 PostgreSQL 14.0 (KesselDB 1.0)
(1 row)

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c "\dt"
Did not find any relations.

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c \
    "CREATE TABLE smoke (id BIGINT, n CHAR(16));"
CREATE TABLE

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c \
    "INSERT INTO smoke (id, n) VALUES (1, 'hello');"
INSERT 0 1

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c "SELECT * FROM smoke;"
 id |   n
----+-------
  1 | hello
(1 row)

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c "\dt"
         List of relations
 Schema | Name  | Type  |  Owner
--------+-------+-------+----------
 public | smoke | table | kesseldb
(1 row)

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c "\d smoke"
              Table "public.smoke"
 Column | Type | Collation | Nullable | Default
--------+------+-----------+----------+---------
 id     | int8 |           |          |
 n      | text |           |          |

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c \
    "INSERT INTO smoke (id, n) VALUES (2, 'world'), (3, 'kessel');"
INSERT 0 2

$ PGPASSWORD=admin psql -h 127.0.0.1 -p 5532 -U test -d kesseldb -c "\dn"
  List of schemas
  Name  |  Owner
--------+----------
 public | kesseldb
(1 row)

What the real-client smoke caught (and SP-PG-CAT-T8 fixed inline):

• BIGINT / INTEGER / SMALLINT / BOOLEAN are now accepted as pure aliases for I64 / I32 / I16 / Bool in CREATE TABLE. Previously a real psql CREATE TABLE foo (id BIGINT) would error with sql: unknown type "BIGINT". INT8 / INT4 / INT2 are NOT aliased because KesselDB's own I8 / I16 / I32 already use those spellings for narrow widths.
• \d <name> is fully supported. psql ships a 5-query catalog walk (OID lookup with OPERATOR(pg_catalog.~) + pg_class relsummary + pg_attribute column list + pg_policy / pg_inherits / pg_trigger / pg_statistic_ext / pg_publication / pg_foreign_table polls); the catalog hook now recognizes every shape, synthesizing the live table description for the column-list and well-framed empties for the V1-absent surfaces (RLS, partitioning, triggers, extended statistics, logical replication, foreign data).
• \dn schema-list is fully supported. Returns the canonical single-row public/kesseldb table.

What the real-client smoke flagged as a known V1 limitation (NOT a catalog bug — these are documented PG-wire query-shape boundaries):

• SELECT n FROM smoke WHERE id = 1; → V1 PG-wire only renders SELECT * FROM <table>. Same for SELECT COUNT(*) FROM smoke. The V1 SELECT-rendering path supports SELECT * from a single table; projected columns + WHERE + aggregates go through the engine SQL layer in V2 SP-PG-EXEC.

Limitations (V1)

Honest scope boundary — V1 PG-wire supports CLI clients (psql, pgcli), programmatic-driver clients (JDBC, psycopg, pgx, tokio-postgres, sqlx-pg), AND GUI admin / BI tools (per the table above). Some advanced introspection paths remain V2-deferred:

• pg_proc real function listing → V1 returns an empty pg_proc so pgAdmin's "Functions" panel is empty + DataGrip's routine browser is empty. V2 SP-PG-CAT-PROC.
• pg_database multi-database → V1 returns one row (kesseldb). A tool that lists databases sees only this one; KesselDB itself has one logical database today. V2 expands when KesselDB grows multi-database (no current plan).
• pg_stat_* runtime statistics → V1 returns zero rows for every pg_stat_* query so prometheus-postgres-exporter reports zero metrics + pgAdmin's "Statistics" tab is empty. V2 SP-PG-CAT-STATS.
• Arbitrary pg_catalog JOIN/GROUP BY/sub-SELECT → V1 recognizes ~35 canonical query patterns the common tools issue. A tool issuing a novel JOIN that doesn't match any pattern still gets 42P01. V2 SP-PG-CAT-AST switches to AST-walking via kessel-sql.
• psql \d+ extended output → V1 covers \d (basic table description); \d+ (with comments + size + stats) is partial (comments + size columns are NULL). V2.
• Cross-schema queries → V1 only knows about public. When KesselDB grows multi-schema (SP-NS), V1 of this arc auto-extends.
• Extended Query SHIPPED at V1.1 (SP-PG-EXTQ, 2026-05-29). Parse / Bind / Describe / Execute / Sync / Close / Flush all dispatched end-to-end. psycopg2 / asyncpg / pgx / tokio-postgres / SQLAlchemy / Drizzle / Prisma / JDBC default-EXTQ paths connect at the wire level. Full ORM-suite formal verification is SP-PG-EXTQ T8 / T11 / T12 (post-V1.1).
• Typed-parameter compile path (SP-PG-EXTQ-PARSED, 2026-06-02). kessel-sql's lexer now recognizes $N (1..99) as a Tok::Param variant; the new compile_with_params(sql, cat, params: &[Option<Value>]) entry point threads typed Values through the parser WITHOUT ever concatenating them into SQL text. Closes the SP-PG-EXTQ V1 §11 weak-spot #1 attack surface (SQL-text-substitution '→'' escaping) for every typed-path-eligible parameter — the bound bytes enter as a typed Value, get carried verbatim through the AST, and emerge in the program as the same Value. Adversarial KAT locked: a quote-injection payload like '; DROP TABLE t; -- in a bound parameter survives as a Value::Blob operand at the EQ comparison; the engine never sees the injected SQL. Gateway-side classifier (preprocess_typed_params) selects the typed path for int/text/bytea/bool params and falls back to the existing text-substitution path for FLOAT/TIMESTAMPTZ/NUMERIC (which still need the cast-wrapper shape 'ISO'::timestamptz). V1 disposition: typed path is opt-in (KAT-only); default remains text-substitution to avoid a silent compat regression. Follow-up SP-PG-EXTQ-PARSED-DEFAULT flips the default after soak.
• Typed-parameter path is now the DEFAULT (SP-PG-EXTQ-PARSED- DEFAULT, 2026-06-02). dispatch_execute routes through EngineApply::apply_sql_with_params(sql, params) whenever the classifier (preprocess_typed_params) returns Some — every bound parameter is then carried as a typed kessel_codec::Value over a new PARAMETERIZED_SQL_TAG = 0xF3 admin frame; the engine thread decodes + runs compile_stmt_with_params against the live catalog. No SQL text concatenation; no '→'' escape rules. Closes the SP-PG-EXTQ V1 §11 weak-spot #1 attack surface at the DISPATCH layer (V1 closed it at the kessel-sql + classifier layer only). Vulcan-verified with psycopg2 + asyncpg + psycopg3 round- trip + quote-injection wire test ("; DROP TABLE inj_smoke; -- stored verbatim, table NOT dropped). Fallback to text-substitute path remains for FLOAT/TIMESTAMPTZ/NUMERIC (parameter shapes the typed path cannot represent cleanly without widening Value).
• BYTEA-binary preserves arbitrary bytes through the typed path (SP-PG-EXTQ-PARSED-BYTEA-TYPED, 2026-06-02). kessel-sql's Tok::Bytes(Vec<u8>) + Lit::Bytes(Vec<u8>) variants thread raw bytes (including non-UTF8 sequences like 0xFF, 0xFE, isolated continuation bytes 0x80-0xBF) from a bound Value::Blob parameter through to the storage layer byte-equal. Drops the prior String::from_utf8_lossy(b).into_owned() call in rewrite_param_tokens that corrupted non-UTF8 bytes before they reached storage. Vulcan-verified with psycopg3 binary-format BYTEA round-trip of payloads fffefd8090a0b0c0, 00...00, and deadbeefcafebabe — all bytes survive verbatim.
• One statement per Q → psql \copy ...; SELECT ... rejected with 42601 syntax_error. Send statements one at a time. V2.
• Text format only → every column rendered as PG text; binary-format preference (advertised in Bind) is rejected with 0A000 feature_not_supported at Bind time. V2 SP-PG-EXTQ-BIN.
• No RETURNING → INSERT ... RETURNING id returns 0A000 feature_not_supported. V2.
• No COPY → \copy users FROM 'data.csv' rejected with 0A000. V2 SP-PG-COPY.
• No LISTEN/NOTIFY → KesselDB has no changefeeds yet. Skip until it does.
• No CancelRequest → V1 emits BackendKeyData (so clients don't refuse to enter the query loop) but ignores incoming CancelRequest on a separate connection. V2 SP-PG T24.
• No TLS → V1 PG-wire is plaintext only. SSLRequest gets the 'N' reply (continue with cleartext). V2 wires rustls behind the existing tls feature gate.
• SCRAM-SHA-256 only → no MD5, no cleartext password, no GSSAPI, no LDAP. Every libpq / JDBC / pgx / psycopg since 2017-2018 supports SCRAM-SHA-256 (PG 10 default), so this is rarely a real-world blocker.
• One credential surface → V1 has ONE shared-secret Bearer token; the PG user field is logged but not authorized against (V2 SP-PG-USERS adds a real user table + per-user privileges).
• SET timezone = … is a no-op → V1 accepts the SET statement (returns CommandComplete: SET) but does not actually rewrite subsequent timestamp formatting. SHOW timezone always returns UTC. V2 wires per-session GUC state.

Troubleshooting

• server closed the connection unexpectedly from psql → KesselDB binary not built with --features pg-gateway, or KESSELDB_PG_ADDR not set, or KESSELDB_TOKEN not set (closed-mode rejects without a token).
• FATAL: invalid_authorization_specification → the Bearer token passed via PGPASSWORD doesn't match KESSELDB_TOKEN. Note: this looks identical to "no token set" on the wire (the no-oracle rule — SCRAM failure modes don't tell the attacker which input was wrong).
• FATAL: sorry, too many clients already (SQLSTATE 53300) → pg_max_conns (default 256) hit. Either close idle clients or raise the cap via ServerConfig.pg_max_conns.
• FATAL: terminating connection due to idle timeout (SQLSTATE 57014) → the connection sent no client message for pg_idle_timeout (default 600s = 10 min). Either reduce session idle time, send a periodic keepalive SELECT 1, or raise pg_idle_timeout for long-lived analytical sessions.
• relation "pg_catalog.pg_proc" does not exist (SQLSTATE 42P01) → V1 of the pg_catalog stubs covers pg_namespace, pg_class, pg_attribute, pg_type, pg_index, pg_constraint + the 5 most-queried information_schema views. pg_proc / pg_stat_* / pg_locks / pg_extension are V2-deferred and remain 42P01 — tools that probe these gracefully degrade (the affected panel is empty but the connection works). See "Limitations (V1)" above for the per-catalog V2 follow-up names.

Real ORM session

Captured from a real Python session driving the kesseldb-server binary (built with --features pg-gateway). Both psycopg2 (libpq Extended Query directly) AND SQLAlchemy 2.0 (higher-level ORM atop psycopg2) round-trip end-to-end, and the matrix extends to psycopg3 / asyncpg / JDBC. The server was started with:

KESSELDB_TOKEN=admin KESSELDB_PG_ADDR=127.0.0.1:5532 \
  ./target/release/kesseldb 127.0.0.1:6532 /tmp/kdb-data
# => KesselDB listening on 127.0.0.1:6532, data dir /tmp/kdb-data, pg=127.0.0.1:5532

Versions: psycopg2 2.9.12 + sqlalchemy 2.0.45 + Python 3.12.3. Total 19 / 19 steps pass on a clean server.

Section 1 — psycopg2 (libpq Extended Query)

import psycopg2
conn = psycopg2.connect(host="127.0.0.1", port=5532,
                        user="test", password="admin", dbname="kesseldb")
conn.autocommit = True
cur = conn.cursor()

# CREATE TABLE + INSERT (parameterized via %s).
cur.execute("CREATE TABLE orm_smoke_t7 (id BIGINT, name CHAR(32))")
cur.execute("INSERT INTO orm_smoke_t7 (id, name) VALUES (%s, %s)",
            (1, "hello"))
cur.execute("INSERT INTO orm_smoke_t7 (id, name) VALUES (%s, %s)",
            (2, "world"))

# SELECT * (no params) + parameterized SELECT WHERE.
cur.execute("SELECT * FROM orm_smoke_t7")
print(cur.fetchall())                # → [(1, 'hello'), (2, 'world')]
cur.execute("SELECT * FROM orm_smoke_t7 WHERE id = %s", (1,))
print(cur.fetchall())                # → [(1, 'hello')]

# DISCARD ALL / STATEMENTS / PORTALS — gateway-intercepted (T7).
cur.execute("DISCARD ALL")
print(cur.statusmessage)             # → 'DISCARD ALL'
cur.execute("DISCARD STATEMENTS")
cur.execute("DISCARD PORTALS")

# BEGIN / COMMIT / ROLLBACK / SET TRANSACTION — gateway-intercepted (T7).
cur.execute("BEGIN")
print(cur.statusmessage)             # → 'BEGIN'
cur.execute("COMMIT")
print(cur.statusmessage)             # → 'COMMIT'
cur.execute("ROLLBACK")
cur.execute("SET TRANSACTION ISOLATION LEVEL READ COMMITTED")
print(cur.statusmessage)             # → 'SET'

# SELECT 1 — SQLAlchemy do_ping() probe (T7 pg_catalog hook).
cur.execute("SELECT 1")
print(cur.fetchall())                # → [(1,)]

cur.close()
conn.close()

Section 2 — SQLAlchemy 2.0

import sqlalchemy as sa

# T8 (2026-05-29) — `use_native_hstore=False` is no longer needed.
# The pg_catalog hook intercepts the canonical psycopg2 hstore-OID
# JOIN probe (`SELECT t.oid, typarray FROM pg_type t JOIN pg_namespace
# ns ON typnamespace = ns.oid WHERE typname = 'hstore'`) and returns a
# 0-row well-framed response, which is the truth — KesselDB has no
# hstore extension.
engine = sa.create_engine(
    "postgresql+psycopg2://test:admin@127.0.0.1:5532/kesseldb",
)

# Full engine.connect() probe sequence + SELECT *.
with engine.connect() as conn:
    rs = conn.execute(sa.text("SELECT * FROM orm_smoke_t7"))
    print(list(rs))                  # → [(1, 'hello'), (2, 'world')]

# Parameterized SELECT via bind-param.
with engine.connect() as conn:
    rs = conn.execute(
        sa.text("SELECT * FROM orm_smoke_t7 WHERE id = :id"),
        {"id": 1},
    )
    print(list(rs))                  # → [(1, 'hello')]

# DISCARD ALL via engine.
with engine.connect() as conn:
    conn.execute(sa.text("DISCARD ALL"))

# Connection-pool checkout/checkin x3 (pool reset triggers DISCARD).
for _ in range(3):
    with engine.connect() as conn:
        list(conn.execute(sa.text("SELECT * FROM orm_smoke_t7")))

T8 — hstore probe now intercepted (no caveat needed)

SQLAlchemy 2.0 + psycopg2 by default queries pg_type for the hstore type OID at connect:

SELECT t.oid, typarray
FROM pg_type t JOIN pg_namespace ns ON typnamespace = ns.oid
WHERE typname = 'hstore'

T8 ships a matcher in pg_catalog::catalog_query_hook that recognizes this canonical psycopg2/SQLAlchemy probe shape (qualified + unqualified forms, mixed qualification, case-insensitive, generic extension typname) and emits a well-framed 0-row response with two OID columns. psycopg2 then concludes "no hstore extension installed" — which is the truth, since KesselDB has no extension catalog — and SQLAlchemy proceeds normally. use_native_hstore=False is no longer required for any modern PG client.

What the smoke test covers — 19/19 PASS

Broader ORM compat matrix

The binary-format parameter path covers the supported PG types (INT2/INT4/INT8/FLOAT4/FLOAT8/BOOL/TEXT/VARCHAR/BYTEA/TIMESTAMPTZ), and the symmetric binary-RESULTS path (DataRow + RowDescription) closes the asyncpg gap. Each row below is an actual driver session.

The binary-format decoder wires into the Bind path: each parameter with format_code=1 (binary) at position i is admitted iff param_oids[i] is one of the V1 supported PG types (INT2/INT4/INT8/FLOAT4/FLOAT8/BOOL/TEXT/VARCHAR/BYTEA/TIMESTAMPTZ), then decoded at Execute time into a SQL literal that flows through the existing substitute layer (bare-int literal for integers, single- quoted + '→''-escaped for text, '\xHEX'::bytea for bytea, etc). Describe('S') synthesizes ParameterDescription from the SQL's $N count when Parse omitted OID hints (V1 emits PG_TYPE_TEXT for each position; clients encode text-as-binary which routes through the existing text path).

SP-PG-EXTQ-BIN-RESULTS T2 then wired the symmetric result-side post-processor: when the portal's Bind requested result_formats=[1] (asyncpg / JDBC default extended mode / sqlx), dispatch_execute re-encodes each buffered DataRow per-column into PG binary format + flips the per-field format_code slot in RowDescription in lockstep so libpq's per-field decoder switches to its binary read path. NULL columns and text-format columns pass through unchanged; the rewrite is zero-cost for the existing text-only path.

SP-PG-EXTQ-BIN-NUMERIC (2026-06-02) — Decimal/BigDecimal round-trip unlocked. The V1 BIN + BIN-RESULTS arcs deferred NUMERIC binary because the PG wire shape is base-10000 variable-length-digit (sign + dscale + weight + N i16 digits) and bug-prone. This follow-up arc ships a pure-Rust NUMERIC codec in crates/kessel-pg-gateway/src/extq/ binary_numeric.rs (decode_numeric_binary + encode_numeric_binary

• BinaryNumericError) covering |value| < 10^18 with ≤18 fractional digits — the typical ORM Decimal/BigDecimal range. Wired into both extq::substitute::decode_binary_param (Bind path) and extq::binary_results::encode_binary_value (Execute result path). The binary_format_supported_for_oid / binary_result_supported_for_oid predicates now include PG_TYPE_NUMERIC (OID 1700). Wider values reject with the precise SP-PG-EXTQ-BIN-NUMERIC-BIGNUM follow-up. NaN / +Infinity / -Infinity (sign=0xC000 / 0xD000 / 0xF000) decode to the canonical PG strings "NaN" / "Infinity" / "-Infinity" and encode from case-insensitive variants of the same strings (including short "inf" / "+inf" / "-inf" aliases) — closed by SP-PG-EXTQ-BIN-NUMERIC-NAN-INF (2026-06-02) at the codec layer; the engine-level NUMERIC storage of these specials remains a separate follow-up (FieldKind::I128 has no native NaN/Inf representation). psycopg2 + asyncpg both decode Decimal('42') / Decimal('-7') / Decimal('999999999') from kesseldb-emitted NUMERIC binary DataRow. COPY binary NUMERIC also works end-to-end — the same codec routes through the COPY-BIN admission + per-row encode/decode paths.

The remaining residual ORM gaps are:

• JDBC simple-query mode hits a kessel-sql parser gap on ::int8 casts → CLOSED 2026-06-02 by SP-PG-EXTQ-CAST T2 — cast_stripper::strip_pg_casts removes ::TYPE[(args)] from SQL text at dispatch_query entry (preserving string/comment context). psql proxy round-trip for SELECT 1::int8 / WHERE id = $1::int8 / INSERT ... VALUES (3::int8, 'x'::text) verified. Real pgJDBC round-trip then verified — JDBC simple-mode WHERE id = 42::int8 round-trips end-to-end through the actual pgJDBC 42.7.4 driver against KesselDB. The cast-stripper is closed end-to-end. Simple-mode PreparedStatement paren- wrapped VALUES → CLOSED 2026-06-02 by SP-PG-SQL-PAREN-VALUES V1 — kessel-sql's VALUES tuple parser now accepts (LITERAL) paren-wrapped literals up to depth 8 (anti-stack-bomb cap at 9 levels), and the same arc adds Str → numeric coercion in the WHERE term parser when the LHS is a numeric column (PG's '42'::int8 semantic preserved across the cast strip). Real pgJDBC simple-mode PreparedStatement INSERT + SELECT WHERE id = ? round-trip end-to-end. Extended-mode SELECT version() Describe/NoData ordering → CLOSED 2026-06-02 by SP-PG-EXTQ-DESCRIBE-VERSION V1 — the gateway's extq::row_description_or_no_data_for_sql helper now recognizes the closed set of scalar SELECTs that SP-PG-EXTQ T7 added Simple-Query handlers for (SELECT version(), SELECT current_user, SELECT 1, etc.) and emits the matching RowDescription at Describe time instead of NoData. pgJDBC extended-mode SELECT version() round-trips end-to-end via real pgJDBC 42.7.4.
• Parameterized SELECT with a CHAR(N) WHERE clause may match zero rows because the engine's EQ-on-Char doesn't ignore trailing NUL padding on the storage side; lifts in SP-CHAR-PAD-COMPARE (engine-side). → CLOSED 2026-06-02 by SP-CHAR-PAD-COMPARE V1 — the engine's kessel-expr EQ / NE / LT / LE / GT / GE opcodes (and the engine-wide kessel-sm::cmp_field helper) now treat trailing NUL (0x00) and space (0x20) as insignificant on Char(_) / Bytes(_) byte comparisons (PG SQL §9.20 semantic, with the storage-aware NUL widening — engine stores fixed-width values NUL-padded). asyncpg WHERE name = $1 against CHAR(32) now returns the matching row; BETWEEN / NE also work; the Describe-on-$N enabler (substitute $N with NULL for the table-name probe) closes the asyncpg ProtocolError that the engine fix unmasked. Storage / indexes / hashing UNCHANGED — only the comparison layer trims. +15 KATs across kessel-expr / kessel-sm / kessel-pg-gateway.
• Binary NUMERIC / JSONB / UUID / ARRAY remain V2 (SP-PG-EXTQ-BIN- NUMERIC / SP-PG-EXTQ-BIN-EXTRA).
• SP-PG-EXTQ-CAST V1 is "strip + hope" — a Bind whose declared param OID disagrees with the SQL's $N::TYPE cast silently coerces. → CLOSED 2026-06-02 by SP-PG-EXTQ-CAST-VALIDATE V1 — cast_stripper::strip_pg_casts_tracked returns (stripped_sql, Vec<($N_index, declared_oid)>); PreparedStmt.param_casts stores the pairs at Parse time; dispatch_bind rejects any mismatch between the bound parameter OID and the declared cast OID with 42846 cannot_coerce. Closes the silent-coercion attack vector the parent arc's "V1 scope is strip + hope" note explicitly flagged. Literal casts (no $N) bypass the validator so the parent arc's psql shapes still PASS.
• SP-PG-EXTQ-CAST-VALIDATE V1 enforces STRICT OID equality — pgJDBC's default Java-int against ::int8 cast (and psycopg3's Python-int against ::int8) false-rejected with 42846 because the wire-supplied INT4 OID didn't equal the declared INT8 OID. → CLOSED 2026-06-02 by SP-PG-EXTQ-CAST-VALIDATE-COMPAT V1 — V1 strict equality relaxes to PG's pg_type.dat::typcategory table. types::oid_category(oid) returns the category byte ('N' numeric, 'S' string, 'B' bool, 'D' date/time, 'U' unknown/bytea); types::oid_castable(param_oid, cast_oid) accepts the pair iff strict equality OR param_oid == 0 (omitted hint skip) OR same-category widening. dispatch_bind's validator swaps the strict != check for !oid_castable(...). Intra-category widenings now accept (INT4↔INT8, INT8↔FLOAT8, INT4↔NUMERIC, TEXT↔VARCHAR, etc.); cross-category mismatches (TEXT vs INT8, BOOL vs INT8, BYTEA vs TEXT) STILL reject with the same ExtqError::CastOidMismatch → 42846 cannot_coerce wire frame so the V1 silent-coercion vector stays closed. Verified via a psycopg3 PQ-layer 5-case smoke: INT4+INT8 / INT8+INT4 / TEXT+VARCHAR all accept; cross-category TEXT+INT8 still rejects with the exact 42846 message; strict- equality INT8+INT8 still works. +14 KATs across types::tests + extq::tests. V2 follow-ups named: SP-PG-EXTQ-CAST-VALIDATE-COMPAT-RANGE (overflow-check the param value vs cast-type range), SP-PG-EXTQ-CAST-VALIDATE-LITERAL (validate literal casts too), SP-PG-EXTQ-CAST-VALIDATE-CATEGORY-CROSS (accept the cross- category casts PG itself accepts, e.g. TEXT '42' → INT8).
• SP-PG-EXTQ-CAST-VALIDATE V1 + COMPAT only validate $N::TYPE casts — a LITERAL::TYPE cast (e.g. SELECT 'hello'::int8) is silently stripped, so a cross-category literal cast that PG would reject slips through whenever the value doesn't reach a typed column. → CLOSED 2026-06-02 by SP-PG-EXTQ-CAST-VALIDATE-LITERAL V1 — cast_stripper::find_literal_cast_mismatch(sql) classifies the literal immediately before each :: (bare integer → INT4/INT8, bare float → FLOAT8, quoted string → TEXT, true/false → BOOL, NULL → anytype) and compares its types::oid_category against the cast type's category. The dispatchers (dispatch_query, dispatch_query_with_params, extq::dispatch_parse) call it BEFORE the strip rewrites the SQL; a cross-category literal cast rejects with 42846 cannot_coerce via the same wire frame the $N validator uses, while NULL::TYPE accepts unconditionally (the canonical typed-NULL idiom). Within-category casts (1::int8, 'hello'::text, true::bool, -1::int8) and strip_pg_casts's byte output are unchanged. Verified via a psql smoke: 1::int8 / 'hello'::text accept; 'world'::int8 (TEXT→INT8) and true::int8 (BOOL→INT8) reject with the literal- cast 42846 message; NULL::int8 is NOT rejected by the validator. +28 KATs (cast_stripper::tests + extq::tests). V2 follow-ups named: SP-PG-EXTQ-CAST-VALIDATE-LITERAL-EXPR (literal casts inside expressions, (1+2)::int8), SP-PG-EXTQ-CAST-VALIDATE-LITERAL-DATEPARSE ('2024-01-01'::date), SP-PG-EXTQ-CAST-VALIDATE-LITERAL-NUMSTR ('42'::int8), SP-PG-EXTQ-CAST-VALIDATE-LITERAL-MULTIWORD (multi-word type names).

Pipelining throughput

Single-statement round-trip throughput measured with psycopg2 (no libpq pipeline mode):

Latency-bound (SOCK_STREAM + Parse/Bind/Execute/Sync flush cost per statement). A libpq-pipeline-mode test would batch up to 8 messages and post higher numbers; that's V2 SP-PG-EXTQ-PIPELINE-BATCH.

SP-PG-COPY — COPY FROM STDIN / COPY TO STDOUT bulk load (V1 SHIPPED 2026-05-30)

PG's COPY command is the bulk-load lever every modern pg_dump restore, sysbench prepare phase, and analyst-friendly psql \copy ... CSV workflow uses — the same wire shape every PostgreSQL-aware ETL tool defaults to. V1 ships text format end-to-end for both directions; CSV + binary deferred to V2 arcs (SP-PG-COPY-CSV, SP-PG-COPY-BIN).

# COPY FROM STDIN — text format, the pg_dump default
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY users FROM STDIN" < users.tsv
# → COPY 1000

# psql \copy is the client-side wrapper around the same wire shape
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  '\copy users FROM /path/to/users.tsv'

# COPY TO STDOUT — text format
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY users TO STDOUT" > users.tsv

# Round-trip: export then re-import produces an identical row set.

# Optional column list works in both directions.
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY users (id, name) FROM STDIN" < partial.tsv
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY users (id, name) TO STDOUT" > partial.tsv

# NULL columns use the PG-canonical `\N` sentinel.
printf '10\tfoo\n20\t\\N\n30\tbar\n' | \
  psql -h kesseldb -p 5532 -U test -d kesseldb -c \
    "COPY t (id, label) FROM STDIN"
# → rows with id=20 have label=NULL

V1 text-format escapes per PG §COPY-FORMATS: rows separated by \n, fields by \t, NULL is \N, the 7 PG-canonical backslash escapes (\b \f \n \r \t \v \\) are recognized on input + emitted on output, and the legacy v2 end-of-data marker \. is tolerated on input.

Connection state model: a COPY ... FROM STDIN Query transitions the connection to CopyIn state — the server then accepts only CopyData, CopyDone, CopyFail, or Terminate until the COPY exchange ends. Any other frontend tag in CopyIn = 08P01 protocol_violation + clean error + state cleared. A subsequent Q or extended-query message works normally — the connection STAYS ALIVE across COPY errors (matching the SP-PG-EXTQ tolerant probe-then-fall-back contract).

Abort-tail drain (SP-PG-COPY-ABORT-DONE-TAIL V1, 2026-06-02): when a per-row error mid-CopyData emits ErrorResponse + RFQ and transitions back to Idle, the client may still be flushing trailing CopyData / CopyDone / CopyFail frames it had already queued before observing the error. Per PG §55.2.7 the server silently drains those tail frames without emitting any additional response. KesselDB does the same: the connection stays alive, the tail bytes are absorbed without a spurious unsupported message tag 08P01, and the next Query (SELECT / COPY / extq Parse / etc.) succeeds on the SAME TCP connection. A stray c / f in pristine Idle with no preceding abort still rejects with 08P01 (defensive shape against a truly broken client). Verified via a psql 16 smoke.

Throughput (100K rows of (BIGINT, CHAR(64))): ~51,840 rows/sec with bulk-apply (default KESSELDB_COPY_BATCH_SIZE=1024). The V1 per-row baseline was ~285 rows/sec; BULKAPPLY V1 lifts 181.9× by folding N rows into a single multi-row INSERT INTO t (cols) VALUES (...), (...), ... which kessel-sql compiles to Op::Txn { ops: Vec<Op::Create> } — one apply round-trip + one WAL fsync per batch instead of one per row. Tunable via KESSELDB_COPY_BATCH_SIZE env at server start (clamped to [1, 65536]); set to 1 to restore V1-baseline shape. Postgres 16 reference on the same workload: ~578K rows/sec — KesselDB is now within ~11× of Postgres COPY throughput (was ~2000× behind).

Atomicity vs PG: SP-PG-COPY-BULKAPPLY V1 is per-batch atomic — each batch (default 1024 rows) is wrapped in an Op::Txn, so any inner-op failure rolls back the whole batch. Real PG is whole-COPY atomic (an implicit transaction wraps every row in the COPY). A constraint failure at row 1500 of 10000 with the default batch size: rows 1-1024 stay committed; rows 1025-1500's batch rolls back; COPY aborts. The named follow-up arc SP-PG-COPY-BULKAPPLY-WHOLECOPY would close the rest of the gap (gated on an engine-side streaming-Txn shape landing first).

NULL-row fallback: a batch containing any \N NULL field falls back to per-row dispatch (the column-omit trick V1 relies on for NULL handling requires per-row column lists, which multi-row INSERT can't carry). Throughput on NULL-heavy tables is therefore similar to the V1 baseline; throughput on all-non-NULL tables (sysbench / pg_dump common case) lands the headline lift.

V1 NULL handling caveat: kessel-sql's INSERT VALUES parser has no NULL keyword. SP-PG-COPY V1 works around this by OMITTING NULL columns from the synthesized INSERT (col, col, ...) VALUES (...) — kessel-sql's SP86 default-fill semantics for omitted nullable columns then applies. This means a NOT NULL column receiving \N surfaces as a clean 23502 not_null_violation error at ingest time (matching PG).

SP-PG-COPY-CSV — CSV format (V1 SHIPPED 2026-06-01)

CSV format unlocks pg_dump --csv + every spreadsheet/analyst on-ramp (Excel, Sheets, R, pandas.read_csv). RFC 4180 grammar with the PG superset (HEADER + custom DELIMITER / QUOTE / ESCAPE / NULL).

# COPY FROM CSV with HEADER (the pg_dump --csv default shape)
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY users FROM STDIN WITH (FORMAT csv, HEADER)" < users.csv
# → COPY 1000  (the header row is skipped)

# COPY TO CSV with HEADER — exports a spreadsheet-openable file
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY users TO STDOUT WITH (FORMAT csv, HEADER)" > users.csv

# Embedded comma / embedded quote / NULL all round-trip byte-equal:
#   1,"Alice, the brave"        ← quoted because of the embedded comma
#   2,"Bob ""the builder"""     ← doubled-quote escape inside the value
#   3,Charlie                   ← bare unquoted
#   4,                          ← empty unquoted = NULL (default)

# Custom delimiter + NULL marker for CSVs exported from Sheets etc.
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY t FROM STDIN WITH (FORMAT csv, HEADER, DELIMITER ';', NULL '<NA>')" \
  < euro-style.csv
# → ';' splits fields; '<NA>' decodes as NULL

V1 CSV codec honors per PG §SQL-COPY CSV defaults: delimiter ,, quote ", escape = quote (so "" is the doubled-quote escape; configure a distinct ESCAPE 'X' to use the alternate single-char escape shape), NULL = empty unquoted (a quoted empty "" stays empty-string, distinct from NULL). HEADER on input consumes the first record; on output emits a first record containing the resolved column names. The CSV record parser is multi-line aware — a quoted field containing literal newlines reassembles correctly across CopyData frame boundaries via the carry buffer.

Rejected CSV options surface precise V2-pointing errors:

ERROR:  COPY csv option FORCE_QUOTE not supported in V1 (SP-PG-COPY-CSV-FORCEQUOTE / SP-PG-COPY-CSV-ENCODING)
ERROR:  COPY csv DELIMITER must be a single character (got '||')

CSV format inherits the SP-PG-COPY-BULKAPPLY V1 batching throughput

• NULL-row fallback semantics — the codec is a payload concern only.

SP-PG-COPY-CSV-NUMERIC — canonical NUMERIC validator (V1 SHIPPED 2026-06-02)

Both text + CSV COPY now validate the canonical PG NUMERIC text grammar at the gateway BEFORE handing the row to the engine, with sign normalisation + case-insensitive NaN/Infinity acceptance:

# Sign normalisation — +999 stored as 999:
echo 'id,amount
1,42
2,12345.6789
3,-3
4,+999' | psql -c "COPY t FROM STDIN WITH (FORMAT csv, HEADER)"

# Case-insensitive NaN / Infinity / -Infinity (validator pass —
# canonicalised to mixed-case PG form before reaching the engine):
#   nan → NaN
#   infinity / +infinity / inf / +inf → Infinity
#   -infinity / -inf → -Infinity

# Malformed input rejects with precise 22P02 + row + column + reason:
psql -c "COPY t FROM STDIN WITH (FORMAT csv)" <<< $'id,amount\n1,1.2.3\n'
# ERROR:  COPY csv row 1 column "amount" NUMERIC: malformed (multiple decimal points)
psql -c "COPY t FROM STDIN WITH (FORMAT csv)" <<< $'id,amount\n1,hello\n'
# ERROR:  COPY csv row 1 column "amount" NUMERIC: bad byte 0x68 at position 0
psql -c "COPY t FROM STDIN WITH (FORMAT csv)" <<< $'id,amount\n1,1e10\n'
# ERROR:  COPY csv row 1 column "amount" NUMERIC: scientific notation not supported in V1 (SP-PG-COPY-CSV-NUMERIC-SCI)

The validator runs on every column whose PG type OID resolves to PG_TYPE_NUMERIC (1700 — kessel-sql I128, U128, Fixed); other column types pass through unchanged. NULL fields are forwarded verbatim (text \N; CSV empty unquoted) so the kessel-sql column-omit auto-NULL-fill semantics keep working.

V1 limitations (each with its own follow-up arc):

• SP-PG-COPY-CSV-NUMERIC-SCI — scientific notation lifted in V1 (see next subsection — 1e10, 1.5e-3, 6.022e23, -3.14e2 now expand cleanly to canonical decimal text in the validator).
• SP-PG-COPY-NUMERIC-BIGNUM — values beyond the kessel-sql i128 (|value| < 10^18) cap surface at INSERT time, not at the validator.
• NaN/Infinity engine storage — the validator accepts and canonicalises NaN/Infinity, but the engine-side I128 literal parser cannot store them yet (engine surfaces sql: expected value). A separate arc lifts the engine-storage gap.

SP-PG-COPY-CSV-NUMERIC-SCI — scientific notation (V1 SHIPPED 2026-06-02)

Both text + CSV COPY now also accept scientific-notation NUMERIC fields and expand the exponent into the canonical PG decimal text BEFORE the row reaches the engine:

# Integer-yielding scientific notation expands cleanly end-to-end:
echo 'id,val
1,1e10
2,6e3
3,-3.14e2
4,1.5e3' | psql -c "COPY t FROM STDIN WITH (FORMAT csv, HEADER)"

# Stored canonical values:
#   1e10      → 10000000000
#   6e3       → 6000
#   -3.14e2   → -314
#   1.5e3     → 1500

# Avogadro-style large exponents in the |exp|<=100 band:
#   6.022e23  → 602200000000000000000000

# Out-of-range exponent (|exp|>100) rejects with precise 22P02:
psql -c "COPY t FROM STDIN WITH (FORMAT csv)" <<< $'id,val\n1,1e1000\n'
# ERROR:  COPY csv row 1 column "val" NUMERIC: malformed (exponent out of range)

# Missing exponent / malformed exponent reject with precise 22P02:
psql -c "COPY t FROM STDIN WITH (FORMAT csv)" <<< $'id,val\n1,1e\n'
# ERROR:  COPY csv row 1 column "val" NUMERIC: malformed (missing exponent)
psql -c "COPY t FROM STDIN WITH (FORMAT csv)" <<< $'id,val\n1,1e1.5\n'
# ERROR:  COPY csv row 1 column "val" NUMERIC: malformed (non-integer exponent)

Grammar accepted: [+-]?(\d+(\.\d+)?|\.\d+)[eE][+-]?\d+ — mantissa (integer or integer+fractional or leading-dot-fractional) + e/E (case-insensitive) + signed integer exponent. Trailing-dot mantissa (5.e2) is the named follow-up arc SP-PG-COPY-CSV-NUMERIC-SCI-TRAILDOT (no ORM/spreadsheet emits it in practice; rejected with the arc name in the 22P02 reason).

The expansion uses a decimal-point-shift algorithm (no bigint dep): the mantissa's digit string is shifted by exp - frac_digit_count places. Leading-zero padding handles 1e-3 → 0.001. Negative- zero canonicalises to 0 (matches V1 -0 → 0 rule).

V2 follow-ups:

• SP-PG-COPY-CSV-NUMERIC-SCI-TRAILDOT — trailing-dot mantissa.
• SP-PG-COPY-NUMERIC-BIGNUM — fractional results from negative-exponent scientific (e.g. 1.5e-3 → 0.0015) pass the validator but the engine-side I128 literal parser only stores integer values; the engine surfaces sql: expected value. Same pre-existing gap V1 documented for NaN/Infinity.

SP-PG-COPY-BIN — binary format (V1 SHIPPED 2026-06-02)

PG binary COPY per §55.2.7 — WITH (FORMAT binary). The wire format every pg_dump --format=custom restore + every JDBC CopyManager.copyIn(PGCopyOutputStream...) + every modern ETL binary-bulk-loader (pg_bulkload, pgloader, Stitch, Fivetran, Airbyte) hard-requires. After this arc shipped, those workflows succeed against KesselDB end-to-end.

# COPY TO STDOUT binary — emits the canonical PGCOPY\n\xff\r\n\0
# signature header + length-prefixed binary values + 0xff 0xff EOD.
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY users TO STDOUT WITH (FORMAT binary)" > users.bin
hexdump -C users.bin | head -2
# 00000000  50 47 43 4f 50 59 0a ff  0d 0a 00 00 00 00 00 00  |PGCOPY..........|
# 00000010  00 00 00 00 02 00 00 00  08 ...

# COPY FROM STDIN binary — round-trips into a fresh table.
psql -h kesseldb -p 5532 -U test -d kesseldb -c \
  "COPY users2 FROM STDIN WITH (FORMAT binary)" < users.bin
# → COPY 3

V1 supports the same 11 column types as SP-PG-EXTQ-BIN + SP-PG-EXTQ-BIN- NUMERIC (param/result binary): BOOL, INT2/INT4/INT8, FLOAT4/FLOAT8, TEXT/VARCHAR, BYTEA, TIMESTAMPTZ, and NUMERIC (the latter shipped at SP-PG-COPY-BIN-NUMERIC V1, 2026-06-02 — reuses the extq::binary_numeric codec the EXTQ-BIN-NUMERIC arc shipped, for |value| < 10^18 with ≤18 fractional digits). Tables with UUID / JSONB / ARRAY columns are pre-rejected at COPY-start with a precise V2-arc-pointing message:

ERROR:  COPY binary: column "data" type OID 3802 not supported in V1 (SP-PG-COPY-BIN-EXTRA)

The binary codec reuses the existing SP-PG-EXTQ-BIN-RESULTS encoder (encode_binary_value) and SP-PG-EXTQ-BIN param decoder (decode_binary_param) verbatim — only the framing layer (copy::binary — 19-byte signature header, per-row length-prefixed field values, 2-byte i16 -1 end-of-data marker) is new. Inherits the SP-PG-COPY-BULKAPPLY V1 batching throughput.

Binary COPY is smoke-tested across all 10 supported types, including NUMERIC round-trip (negative values + byte-equal re-export md5 match).

Rejected variants surface precise V2-pointing error messages:

ERROR:  COPY FROM/TO file path not supported in V1; use STDIN/STDOUT (SP-PG-COPY-FILE)
ERROR:  COPY FROM/TO PROGRAM not supported (permanent security restriction)

V2 follow-ups (each its own SP-arc):

• SP-PG-COPY-BIN-NUMERIC — CLOSED at V1 (2026-06-02) — binary NUMERIC routes through the SP-PG-EXTQ-BIN-NUMERIC codec.
• SP-PG-COPY-BIN-OID — the optional OID-column flag bit (legacy PG ≤11 WITH OIDS tables).
• SP-PG-COPY-BIN-EXTRA — binary UUID / JSONB / ARRAY encoding.
• SP-PG-COPY-BIN-DIRECT — bypass the per-value binary→text round trip with typed parameter binding (5-10× throughput win for binary- heavy workloads).
• SP-PG-COPY-CSV-FORCEQUOTE — FORCE_QUOTE (cols) / FORCE_NOT_NULL / FORCE_NULL column-scoped CSV modifiers.
• SP-PG-COPY-CSV-ENCODING — non-UTF-8 CSV input/output encodings.
• SP-PG-COPY-CSV-HEADER-MATCH — PG-15+ HEADER MATCH (validate input header against table schema).
• SP-PG-COPY-BULKAPPLY-WHOLECOPY — whole-COPY atomicity (one Op::Txn covers every row) for full PG-compatible all-or-nothing semantics. Gated on an engine-side streaming-Txn shape (Op::TxnBegin / TxnAppend / TxnCommit) landing first; otherwise a 100M-row COPY would buffer 100M rows in RSS.
• SP-PG-COPY-BULKAPPLY-NULLBATCH — restore the BULKAPPLY win for batches containing NULL fields (today they fall back to per-row dispatch).
• SP-PG-COPY-FILE — COPY ... FROM '/path' (operator-opt-in only, security).
• SP-PG-COPY-PROGRAM — COPY ... FROM PROGRAM '...' (permanent hard pass).

10. HTTP gateway

Opt-in HTTP/1.1 surface (plus a WebSocket upgrade — see §10.5 below) for operators, browsers, and tools that prefer HTTP/JSON over the binary wire protocol. Built with cargo build --release -p kesseldb-server --features http-gateway (add ,tls for HTTPS). The binary wire protocol is byte-untouched and remains the default + fast path; the gateway runs on a sibling TCP listener.

Configuration

#![allow(unused)]
fn main() {
let cfg = kesseldb_server::ServerConfig {
    http_addr: Some("127.0.0.1:6789".parse().unwrap()),
    http_tls_addr: Some("127.0.0.1:6790".parse().unwrap()), // requires `tls`
    tls: Some((cert_pem.into(), key_pem.into())),           // requires `tls`
    token: Some(b"my-token".to_vec()),                      // optional Bearer
    ..Default::default()
};
}

Routes

Auth

In token mode (ServerConfig.token == Some(...)), every request must carry Authorization: Bearer <token> (constant-time compared, RFC 6750 §2.1 case-insensitive scheme). In open mode the header is ignored. Mismatched or missing in token mode → HTTP 401 with {"status":"unauthorized"}.

Exactly-once (optional)

Add the headers X-Kessel-Client-Id: <32-char lowercase hex u128> and X-Kessel-Req-Seq: <decimal u64> together to bind the request to the engine's per-client dedup map — retrying the same (client_id, req_seq) returns the cached OpResult. Both-or-neither (one alone → 400). Duplicate Authorization / X-Kessel-Client-Id / X-Kessel-Req-Seq headers are rejected at parse-time per the exactly-once contract.

curl examples

# Health
curl -s http://127.0.0.1:6789/v1/health
# → {"status":"ok","primary":true,"view":0,"op_number":42,"role":"primary"}

# SQL
curl -s -X POST --data-binary 'CREATE TABLE t (v U64 NOT NULL)' \
  -H 'Content-Type: text/plain' \
  http://127.0.0.1:6789/v1/sql
# → {"status":"ok"}

# Metrics (for Prometheus scrape)
curl -s http://127.0.0.1:6789/v1/metrics
# → # HELP kesseldb_ops_total Number of Ops applied since process start.
#   # TYPE kesseldb_ops_total counter
#   kesseldb_ops_total{kind="applied"} 1234
#   ...

# Token mode
curl -s -H 'Authorization: Bearer my-token' \
  http://127.0.0.1:6789/v1/health

Error mapping (excerpt — full table in spec §4.4)

Full mapping: docs/superpowers/specs/2026-05-24-kesseldb-http-gateway-design.md §4.4.

Prometheus metrics (bounded cardinality)

• kesseldb_ops_total{kind="applied"} — counter
• kesseldb_inflight — gauge
• kesseldb_last_op_number — gauge
• kesseldb_view_number — gauge
• kesseldb_is_primary — gauge (0 or 1)
• kesseldb_http_requests_total{path,status} — counter (V1: empty; wiring in follow-up)

WebSocket gateway (SP-WS)

The HTTP gateway includes a WebSocket arm at GET /v1/ws for long-lived push/streaming clients that don't want a per-request HTTP round trip. Shipped under SP-WS, lives in the same kessel-http-gateway crate, enabled automatically by --features http-gateway. There is no separate ws-gateway feature flag.

Wire shape

• RFC 6455 strict handshake. Required headers: Upgrade: websocket, Connection: Upgrade, Sec-WebSocket-Version: 13, Sec-WebSocket-Key, and Sec-WebSocket-Protocol: kessel-op-v1. Server replies with the matching Sec-WebSocket-Accept (RFC 6455 §4.2.2 SHA-1 / base64) and echoes Sec-WebSocket-Protocol: kessel-op-v1.
• Binary frames only. Each frame payload is one Op::encode() request; the server replies with one OpResult::encode() per request.
• Bounded send queue (16 messages). A slow client cannot grow the server send buffer unbounded — the session closes if the queue is full at enqueue time.
• 30 s ping/pong heartbeat. If the peer fails to respond to a Ping within the deadline the session closes with a 1011 internal error.
• Idle timeout (default 30 s with no inbound message) → graceful close handshake (Close 1000).
• Subprotocol kessel-op-v1 is required; clients that omit it are rejected with HTTP 426 upgrade_required. JSON-over-WS is a V2 follow-up.

Auth

Same Bearer token as HTTP, checked once at handshake via the standard Authorization: Bearer <token> header. After the upgrade succeeds the session is trusted for its lifetime — there is no per-frame auth replay. Rotating ServerConfig.token invalidates every future handshake (existing WS sessions keep running until they close).

Backpressure & limits

WS sessions share the engine's max_inflight cap with HTTP and the binary protocol. Per-session bounds: 16-message send queue, max frame payload = http_max_body (default 8 MiB), strict RFC 6455 framing (rejects RSV1-3 bits, masked server-to-client frames, fragmented control frames).

Browser example

const ws = new WebSocket('ws://127.0.0.1:8080/v1/ws', 'kessel-op-v1');
ws.binaryType = 'arraybuffer';
ws.onopen = () => {
  // Send a binary Op::encode() frame
  ws.send(encodeOp({ kind: 'Select', table: 't' }));
};
ws.onmessage = ev => {
  const result = decodeOpResult(new Uint8Array(ev.data));
  console.log(result);
};

HTTP + WS spec + design

• HTTP gateway spec: docs/superpowers/specs/2026-05-24-kesseldb-http-gateway-design.md
• SP-WS WebSocket spec: docs/superpowers/specs/2026-05-26-kesseldb-spws-websocket-design.md
• Internal record: docs/superpowers/specs/2026-05-24-kesseldb-subproject141-http-gateway.md

11. Deploying to the cloud

KesselDB ships four supported deploy shapes — pick the one that matches the runtime you already operate. The V1 cloud-deploy story is single-pod / single-VM (matches the engine's single-writer posture); replicated VSR clustering on k8s / Fly.io is a roadmap follow-up.

11.1 Docker (single-host)

The fastest path from zero to a running node:

docker run --rm \
  -p 6532:6532 -p 6533:6533 -p 5432:5432 \
  -v $PWD/kesseldb-data:/data \
  -e KESSELDB_TOKEN=$(openssl rand -hex 32) \
  ghcr.io/hassard0/kesseldb:latest

The image is multi-arch (linux/amd64 + linux/arm64), runs as a dedicated non-root kessel:1100 UID, exposes all three wire surfaces, and is ~77 MiB stripped. See Dockerfile for the layout + env var matrix.

11.2 Kubernetes (Helm chart)

A Helm chart lives at deploy/helm/kesseldb/. Single-pod, ReadWriteOnce PVC, ClusterIP service.

# 1. Pre-create the token Secret (Helm chart references it by name).
kubectl create secret generic kesseldb-token \
  --from-literal=token=$(openssl rand -hex 32)

# 2. Install.
helm install kesseldb ./deploy/helm/kesseldb

# 3. Verify.
kubectl wait --for=condition=ready pod -l app=kesseldb --timeout=120s
kubectl exec deploy/kesseldb -- \
  kessel --addr 127.0.0.1:6532 --token "$KESSELDB_TOKEN" 'SELECT 1'

Overridable values (full list in deploy/helm/kesseldb/values.yaml): image.tag, persistence.size, persistence.storageClassName, resources.{requests,limits}, service.type, auth.secretName (set to "" for open mode).

The Helm chart is tested end-to-end in CI.

11.3 Fly.io (fly.toml)

A ready-to-deploy fly.toml lives at deploy/fly/fly.toml.

cd deploy/fly
fly launch --no-deploy --copy-config --name <your-app>
fly secrets set KESSELDB_TOKEN=$(openssl rand -hex 32)
fly volumes create kesseldb_data --size 10 --region iad
fly deploy

Full walkthrough + connect-from-outside section + backup commands in deploy/fly/README.md.

11.4 Custom (any container runtime)

The Docker image is a plain OCI image — anywhere that runs OCI containers (Nomad, ECS, Cloud Run, Azure Container Apps, your own systemd-nspawn unit) works the same way:

image:      ghcr.io/hassard0/kesseldb:latest
entrypoint: /usr/local/bin/kesseldb
args:       ["0.0.0.0:6532", "/data"]
env:        KESSELDB_TOKEN  (required for auth; omit for open mode)
            KESSELDB_HTTP_ADDR=0.0.0.0:6533  (opt-in HTTP gateway)
            KESSELDB_PG_ADDR=0.0.0.0:5432    (opt-in PostgreSQL gateway)
volume:     /data  (mount a persistent volume here; the engine writes
                    its WAL + LSM + manifest under this dir)
ports:      6532/tcp  (binary protocol)
            6533/tcp  (HTTP/1.1 + WebSocket — opt-in)
            5432/tcp  (PostgreSQL Frontend/Backend v3.0 — opt-in)

Health check: TCP probe on :6532 is sufficient (engine being up implies the surface is accepting connections). If you have the HTTP gateway enabled, prefer GET /v1/health (returns {"status":"ok","primary":true,...} from the active engine).

11.5 Kubernetes cluster mode

Replicated VSR consensus (kessel-vsr, 3 or 5 replicas) under a single Helm install — survives primary-pod kill + view-change + elects a new primary without operator intervention.

Opt-in via --set cluster.enabled=true:

# 1. (open mode here — set auth.secretName for token auth)
helm install kesseldb-cluster ./deploy/helm/kesseldb \
  --set cluster.enabled=true \
  --set cluster.replicas=3 \
  --set auth.secretName=

# 2. Wait for every replica to be Ready.
kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=kesseldb \
  --timeout=120s
# kesseldb-cluster-0 / -1 / -2 all Running.

# 3. The cluster runs as a StatefulSet with stable pod DNS:
#      kesseldb-cluster-<idx>.kesseldb-cluster-headless.<ns>.svc.cluster.local
#    Peer traffic uses port 6534 (the headless Service publishes
#    only that port); client traffic uses 6532 on the regular
#    ClusterIP Service.

# 4. Talk to the cluster via the failover-aware kessel CLI.
#    --addrs takes a comma-separated address list; the CLI rotates
#    past any node that answers UNAVAILABLE and lands the SQL on
#    the active primary.
ADDRS=kesseldb-cluster-0.kesseldb-cluster-headless.default.svc.cluster.local:6532,\
      kesseldb-cluster-1.kesseldb-cluster-headless.default.svc.cluster.local:6532,\
      kesseldb-cluster-2.kesseldb-cluster-headless.default.svc.cluster.local:6532

kubectl exec kesseldb-cluster-0 -- kessel --addrs "$ADDRS" \
  'CREATE TABLE acct (id BIGINT NOT NULL, bal BIGINT NOT NULL)'
kubectl exec kesseldb-cluster-0 -- kessel --addrs "$ADDRS" \
  'INSERT INTO acct ID 1 (id, bal) VALUES (1, 100)'
kubectl exec kesseldb-cluster-0 -- kessel --addrs "$ADDRS" \
  'SELECT SUM(bal) FROM acct'
# = 100  (16 bytes)

Primary-kill failover:

# Identify the current primary from logs.
for p in 0 1 2; do
  echo "--- kesseldb-cluster-$p ---"
  kubectl logs kesseldb-cluster-$p | grep -i "elected primary" | tail -1
done
# kesseldb-cluster-0: kesseldb cluster: replica 0 elected primary (view=0)

# Kill it.
kubectl delete pod kesseldb-cluster-0 --grace-period=1

# Within view-change timeout (~5s default) a surviving replica is elected.
sleep 8
kubectl logs kesseldb-cluster-1 | grep -i "elected primary" | tail -1
# kesseldb-cluster-1: kesseldb cluster: replica 1 elected primary (view=1)

# Issue another write via --addrs — the CLI rotates past the deleted
# primary's address and lands on the new primary.
kubectl exec kesseldb-cluster-1 -- kessel --addrs "$ADDRS" \
  'INSERT INTO acct ID 2 (id, bal) VALUES (2, 200)'
# OK

kubectl exec kesseldb-cluster-1 -- kessel --addrs "$ADDRS" \
  'SELECT SUM(bal) FROM acct'
# = 300  (16 bytes)   ← 100 + 200, the committed total

Cluster mode and primary-kill failover are tested end-to-end in CI.

Overridable values for cluster mode (full list in deploy/helm/kesseldb/values.yaml): cluster.enabled, cluster.replicas (3 or 5), cluster.peerAddressTemplate, cluster.viewChangeTimeout, cluster.peerPort (default 6534), cluster.podManagementPolicy (default Parallel).

Prometheus monitoring

The chart can emit prometheus-operator CRDs (monitoring.coreos.com/v1 ServiceMonitor + PrometheusRule) that point your Prometheus install at KesselDB's /v1/metrics endpoint and ship three canned alerts on the cluster failure modes that matter. The CRDs are OFF by default — the chart installs cleanly in clusters without prometheus-operator — opt in with --set monitoring.prometheus.enabled=true:

helm upgrade kesseldb-cluster ./deploy/helm/kesseldb \
  --set cluster.enabled=true \
  --set cluster.replicas=3 \
  --set monitoring.prometheus.enabled=true \
  --set monitoring.prometheus.additionalLabels.release=prometheus
# The `release=prometheus` label matches the default kube-prometheus-stack
# ServiceMonitor selector. Skip it if your operator selects on
# something else.

# Verify the CRDs landed:
kubectl get servicemonitor,prometheusrule -l app.kubernetes.io/instance=kesseldb-cluster
#   servicemonitor.monitoring.coreos.com/kesseldb-cluster   30s   <created>
#   prometheusrule.monitoring.coreos.com/kesseldb-cluster   30s   <created>

The ServiceMonitor scrapes the chart's client ClusterIP Service on the named http port (6533) at /v1/metrics. The PrometheusRule ships four alerts:

Emitted metrics (from crates/kessel-http-gateway/src/metrics_writer.rs in single-node mode, and from crates/kesseldb-server/src/cluster.rs render_cluster_metrics_text in cluster mode):

Plus the Prometheus-injected up{} per scrape target. view_changes_total is per-process and resets on replica restart — Prometheus's rate() handles counter resets explicitly via its reset-detection algorithm, so the ViewChangeStorm alert remains correct across restart windows. replica_lag_opnum accuracy is bounded by Prepare-message cadence — a quiet primary leaves the gauge stale at the last Prepare's op_number (accurate within one 12 ms tick under load; stale during quiet).

Knobs (all under monitoring.prometheus.*): enabled, interval (default 30s), scrapeTimeout (default 10s), additionalLabels (ServiceMonitor object labels — used by some operator releases to select which ServiceMonitors to honour), rules.enabled (default true — set false to scrape WITHOUT the canned alerts), rules.additionalLabels.

Cluster-mode V1 limits (named, not vague):

• HTTP/WS/PG-wire SQL/Op gateways NOT served in cluster mode V1. The cluster path serves the binary client protocol on the client port, plus a dedicated metrics-only HTTP endpoint (/v1/metrics + /v1/health, no SQL or Op surfaces) bound to KESSELDB_HTTP_ADDR for Prometheus scrape and liveness checks. Full HTTP/WS/PG SQL/Op gateway surfaces in cluster mode remain a documented V2 follow-up.
• Cross-node exactly-once on SQL writes is NOT guaranteed. The CLI's --addrs retry uses [0xFE] ++ sql (the same wire as single-target SQL) which the cluster server's apply_raw path accepts on every node. For STRICT cross-node exactly-once on writes (replay a committed SQL after a primary kill returns the cached reply rather than re-executing), embed ClusterClient directly and call the Op-level session-framed call(&Op) surface — the same shape the cluster KATs use.
• Fly.io multi-region cluster deploy NOT in V1. Fly Machines don't have stable headless-Service-style DNS; per-region cluster deploy uses a different transport (<machine-id>.vm.<app>.internal). Named V2 follow-up: SP-Cloud-Cluster-GEO (multi-region) and the upstream SP-Cloud-Cluster T6 Fly slice (single-region, 6PN address mesh).
• Online cluster reconfiguration (add/remove replicas without restart) NOT in V1. Static N is the V1 contract. Named V2 follow-up: SP-Cloud-Cluster-RECONFIG (requires upstream kessel-vsr membership-change support).
• Coordinated cluster-wide backup NOT in V1. Per-pod PVC snapshots are uncoordinated (every replica has every byte, so any one snapshot is recoverable; a quiesce-at-op-number cluster-wide snapshot is a separate design). Named V2 follow-up: SP-Cloud-Cluster-BACKUP.

V1 cloud-deploy caveats (named, not vague)

• Single-pod / single-VM by design Cluster mode shipped (V1). SP-Cloud-Cluster T3+T5 lands a 3 or 5 replica StatefulSet + headless Service + per-replica PVC + failover-aware kessel --addrs ... CLI. Multi-region (cross-zone WAN-tolerant view-change) is the named SP-Cloud-Cluster-GEO follow-up; sharding × clustering is SP-Cloud-Cluster-SHARD.
• No public TLS in the v1 ghcr.io image. The image is built with --features pg-gateway,http-gateway only; --features tls is opt-in (rustls). Pair with your platform's ingress (k8s Ingress + cert-manager, fly certs, etc.) or a fronting reverse proxy if you need HTTPS in front of :6533 from the public internet.
• GHCR package visibility. The ghcr.io/hassard0/kesseldb package is currently private (default for new GHCR packages); to pull from a fresh cluster without imagePullSecrets, flip the package to Public in the GitHub UI (repo Packages -> kesseldb -> Settings -> Change visibility -> Public).

12. Backup & monitoring

Both are handled on the engine thread, so a snapshot is crash‑consistent and metrics are exact. Using the embedded engine handle:

#![allow(unused)]
fn main() {
let engine = kesseldb_server::spawn_engine("./data")?;

// Hot, consistent snapshot — recovers to the exact live state digest:
engine.snapshot("./backup-2026-05-17")?;

// Live metrics:
let s = engine.stats();   // ServerStats { applied_ops, digest, uptime_secs }
}

StateMachine::open("./backup-...") recovers an identical state. The digest field matches Replica::digest, so comparing stats across a cluster detects replica divergence. In a cluster, Node::probe() returns (digest, op_number, commit) for the same purpose.

Restore = point a fresh node at a snapshot directory and start it.

13. Wire protocol

Each message is length‑prefixed: [u32 little‑endian length][payload].

This is intentionally tiny — any language can speak it with a socket and the length framing. kessel-client implements all of it.

14. Troubleshooting

For internals see docs/ARCHITECTURE.md; for exactly what is proven vs. roadmap and the performance log see docs/STATUS.md.

CLI — the kessel command-line client

The kessel binary is the line-oriented, JSON-capable client. It is the fastest path for humans, scripts, ops, and agents — meaningful exit codes (0 ok / 1 statement-or-connection error / 2 bad usage) mean you do not have to scrape text to detect success.

See the full reference in Usage guide (full) §2b. The kessel command-line client — interactive shell commands, --json mode, pipe a .sql file, --addr/--token remote auth.

For the binary wire protocol underneath, see Wire protocol.

SQL surface

KesselDB compiles SQL server-side against the live catalog. Supported surface (each item is covered by the test suite):

• DDL — CREATE TABLE, ALTER TABLE … ADD COLUMN (online, no lock), DROP TABLE, CREATE [UNIQUE|RANGE] INDEX, DROP INDEX, DESCRIBE, EXPLAIN.
• DML — INSERT … VALUES (…),(…)* (multi-row = one atomic op), UPDATE, DELETE.
• Queries — SELECT * | <projection> with WHERE (=, !=, <, <=, >,

  =, AND/OR/NOT, IN, BETWEEN, LIKE, IS [NOT] NULL), JOIN, GROUP BY, ORDER BY, LIMIT/OFFSET, COUNT/SUM/MIN/MAX/AVG.

• Constraints — NOT NULL, UNIQUE, foreign keys (ON DELETE RESTRICT/CASCADE/SET NULL), CHECK, deterministic triggers, deterministic WASM-MVP UDFs.
• Transactions — SQL BEGIN/COMMIT/ROLLBACK (atomic non-interactive write batch; reads inside BEGIN are rejected by design) plus op-level Op::Txn for the cluster path.

Full reference (every form, every keyword, every operator): Usage guide (full) §4–§6.

HTTP gateway

Opt-in HTTP/1.1 surface for operators, browsers, and tools that prefer HTTP/JSON. Build with --features http-gateway; add ,tls for HTTPS.

Routes: /v1/sql, /v1/op, /v1/health, /v1/metrics (Prometheus text v0.0.4). Authorization is Bearer with constant-time comparison; exactly-once headers X-Kessel-Client-Id + X-Kessel-Req-Seq are both-or-neither. Full route table, status-code mapping, Prometheus metric names, and curl examples: Usage guide (full) §10.

The binary wire protocol on the primary port is byte-untouched whether the HTTP gateway runs or not.

WebSocket gateway

GET /v1/ws upgrade, kessel-op-v1 subprotocol, RFC 6455 strict handshake. Each binary frame is one Op::encode() request; the server replies one OpResult::encode(). Bounded send queue (16 messages), 30 s ping/pong heartbeat, 30 s idle close. Bearer auth checked once at handshake.

Ships under the same --features http-gateway flag — there is no separate ws-gateway feature; the WS arm lives in the same kessel-http-gateway crate.

Browser example, full wire-shape spec, backpressure model: Usage guide (full) §10 → WebSocket.

PostgreSQL wire

KesselDB speaks PostgreSQL Frontend/Backend Protocol v3.0 (Simple Query path + SCRAM-SHA-256). Built behind --features pg-gateway.

The operator's Bearer token IS the SCRAM password — one credential surface; rotating the token rotates HTTP + WS + PG-SCRAM atomically.

Supported clients (each verified via synthetic-peer KATs that replay the tool's verbatim connect / introspection SQL):

• CLI — psql, pgcli
• Drivers — org.postgresql:postgresql JDBC, psycopg2/psycopg3, pgx, tokio-postgres, sqlx-pg
• GUI / BI — pgAdmin 4, DBeaver, DataGrip / IntelliJ, Metabase, Tableau, Looker, Hex, Superset
• ORM — Drizzle, Prisma, GORM, Diesel (simple-query mode; Extended Query / prepared statements is V2 SP-PG-EXTQ)

Real session capture, the supported pg_catalog / information_schema surface (SP-PG-CAT), V1 limitations, and troubleshooting: Usage guide (full) §9.

External sources & Parquet

Register a named table whose rows are populated from a remote JSON/NDJSON/CSV/Parquet endpoint or an S3-compatible / Azure Blob object, then query it with ordinary SQL.

• HTTP / HTTPS sources — --features external-sources (HTTP) or --features external-sources-tls (adds HTTPS via rustls).
• Object-store sources — --features external-sources-objstore (implies external-sources-tls); supports s3:// and az://.

The pure-Rust zero-dep Parquet reader supports the full pyarrow 24.0.0 matrix (`UNCOMPRESSED + Snappy + GZIP + zstd + LZ4_RAW + Brotli × PLAIN

• dictionary × V1 + V2 pages × flat REQUIRED + OPTIONAL + LIST + MAP<K,V> + struct + 3-deep cross-products × INT32 + INT64 + INT96 + DECIMAL(≤38) + FLBA + BYTE_ARRAY`).

Reference: Usage guide (full) §7c–7f. The Parquet capability matrix lives in README.md.

Running a cluster

A cluster is composed from the kesseldb-server library — each node runs the deterministic engine wrapped in a Viewstamped Replication replica; nodes talk over real TCP sockets. ClusterClient rotates the address list and retries on Unavailable with stable (client, req) exactly-once semantics.

For horizontal scale, run K independent VSR shard groups behind a router — a single-shard transaction stays on its shard's own group (serializable, fast path); cross-shard transactions are deterministic (Calvin-style, no 2PC) via a replicated sequencer group.

Full reference (code snippets, peer-address layout, recovery, the deterministic Calvin-style cross-shard transaction model): Usage guide (full) §7 + §7b.

Architectural background: Architecture → MVCC & VSR and Architecture → Sharding.

Authentication & TLS

KesselDB has one credential surface: a shared-secret Bearer token, compared in constant time. The same token authorizes the binary wire (0xFC handshake), HTTP (Authorization: Bearer …), WebSocket (handshake-time), and the PostgreSQL wire (the token IS the SCRAM password input). Rotating the token rotates every wire surface at once.

TLS is opt-in to preserve the zero-dependency default:

• --features tls — terminate TLS for the binary wire (rustls).
• --features http-gateway,tls — terminate HTTPS on ServerConfig.http_tls_addr (same rustls config).
• --features external-sources-tls — allow https:// external sources (rustls + bundled Mozilla webpki roots; full certificate + hostname verification, no bypass).
• --features external-sources-objstore — implies external-sources-tls; every s3:// and az:// request is HTTPS-only.

The default plaintext binary wire is token-authenticated; deploy behind a TLS-terminating reverse proxy, or on a private network (WireGuard, tailnet, VPC) if you don't want the rustls dependency.

Full reference (auth config, connection quotas, backpressure): Usage guide (full) §8.

Backup & monitoring

Backup and metrics both run on the engine thread, so a snapshot is crash-consistent and metrics are exact.

• Hot consistent snapshot — engine.snapshot("./backup-DATE")?. Recover with StateMachine::open("./backup-DATE"); the recovered digest matches the live one byte-for-byte.
• Live metrics — engine.stats() returns ServerStats { applied_ops, digest, uptime_secs }. In a cluster, Node::probe() returns (digest, op_number, commit) so you can detect replica divergence by comparing across nodes.
• Prometheus — with --features http-gateway, scrape /v1/metrics for kesseldb_ops_total / kesseldb_inflight / kesseldb_view_number / kesseldb_last_op_number / kesseldb_is_primary (cardinality bounded by design).

Full reference: Usage guide (full) §11.

Troubleshooting

Common symptoms and the underlying cause:

Full table including the HTTP gateway error map and the Parquet typed-Unsupported messages: Usage guide (full) §13 and §9 Troubleshooting.

Architecture overview

The full architecture document, in-place. The sub-chapters that follow (Determinism seam, MVCC & VSR, Sharding, Wire protocols) are landing pages that point at the relevant section of this overview.

KesselDB Architecture

This document is the internals reference. It assumes you've read README.md (the front door) and at least skimmed docs/USAGE.md (operator + SQL reference).

The structure goes inside-out: foundational seams → replication + sharding → storage + MVCC → SQL surface → wire protocols → rigor artifacts → limitations. Every named subsystem has a separate progress / design spec under docs/superpowers/specs/; this doc summarizes and links — it does not substitute for the per-slice spec when you're modifying a subsystem.

The determinism seam

Everything above storage is a pure function over an injected clock, disk, and network (kessel-io). Production injects real I/O; kessel-sim injects a seeded, fault-injecting fake. The whole database runs deterministically from one u64 seed — this is what makes a from-scratch VSR reimplementation verifiable rather than hopeful.

kessel-sm, kessel-catalog, kessel-codec contain zero I/O / clock / RNG.

Crates

Kernel (default cargo build, zero external dependencies):

Optional (feature-gated, still zero external runtime deps):

kessel-parquet decodes every nested shape pyarrow writes up to 3-deep nesting (List, Map, struct, and all cross-products). Supported codecs: Uncompressed, Snappy, GZIP, zstd, LZ4_RAW, Brotli (6 of 7; legacy LZ4 framing remaining). Page versions V1 + V2; encodings PLAIN + RLE_DICTIONARY. The assembly module hosts the Dremel record assemblers. See the OBJ-2c-* design specs for shape coverage.

Replication (VSR)

Viewstamped Replication ported from TigerBeetle's design:

• Primary assigns op-number + a deterministic timestamp
• Prepare → f+1 PrepareOk → Commit
• Backups apply in op-number order
• View-change on primary timeout; state transfer for lagging replicas
• Client table for exactly-once retried client batches

Fixed cluster size (3 or 5); membership reconfiguration is out of scope for v1. Correctness is mechanically verified via TLA+ (Replication.tla, 528M distinct states, depth 21, 0 violations) and exercised by the seeded VSR corpus including the historically-hard seed 7 (see SP46 for the diagnosis + fix that closed seed 7 liveness).

Sharding & cross-shard transactions

Deterministic key→shard mapping (rendezvous hashing over the 20-byte row key). A deployment runs K independent VSR shard groups behind a router: a request goes to the shard that owns its key; schema / DDL is broadcast to every shard (identical catalogs ⇒ deterministic per-shard execution); a single-shard transaction stays on that shard's own VSR group (serializable, fast path).

Cross-shard transactions are deterministic (Calvin-style), not 2PC. A cross-shard Op::Txn is decomposed into per-shard slices and a descriptor is durably totally ordered by a dedicated replicated sequencer group (an ordinary VSR cluster; one append op assigns a gap-free seq, the counter lives in the digest). Each shard then processes every global seq in order via two phases:

• decide — dry-run the slice against committed state and persist a stable verdict (applies nothing)
• commit — apply the slice iff the global decision (the AND of participant verdicts) is commit, else a deterministic atomic skip; the per-shard cursor advances lockstep with the global order

Because every verdict is a pure function of that group's durable state, the global decision is recomputable by any router (no coordinator whose crash loses the outcome) and no locks are held across shards. Properties: atomic (all-or-none across shards), exactly-once under client retry (stable (client, req) keying with a digest-resident dedup map), and recoverable (a full ordered re-drive after a router restart is idempotent — verdicts are stable, commits cursor-idempotent).

Cross-shard reads (SP-A scatter scan)

Op::Select / Op::QueryRows / Op::SelectFields / Op::SelectSorted fan out across every shard via the router-side scatter_scan helper in crates/kesseldb-server/src/scatter_scan.rs. The client wire stays unchanged — the router translates each scan into K parallel per-shard calls and merges the per-shard OpResult::Got(...) payloads into a single byte-shaped result.

Fan-out is zero-dep std-thread: one std::thread per shard, each driving a per-shard ClusterClient (the ShardCaller trait); per-shard reply channels are bounded sync_channel(SHARD_BACKPRESSURE_BOUND=4) for skew defense.

Merge has two strategies discriminated by ScatterKind:

• Unordered (Select / QueryRows / SelectFields): shard-id-ordered concatenation of every shard's [u32 rowlen][record]* payload, truncated to LIMIT rows. Order is deterministic, not arrival-order, so the merged bytes are replay-safe.
• Sorted (SelectSorted): k-way BinaryHeap merge of per-shard already-sorted streams; OFFSET + LIMIT are applied in the merge loop, not shard-side.

LIMIT cancellation uses a shared Arc<AtomicBool> flag fired the instant output.len() == LIMIT; late shards see it pre-call so the router isn't pinned waiting.

K-invariance (the headline correctness property): for SelectSorted with unique sort values, the merged output is byte-identical to a K=1 baseline for K ∈ {1, 2, 4, 8, 16}. Locked by SP-A T3's 425-fixture property sweep at the merge layer plus a real-socket K=1↔K=4 integration test.

See "Limitations & V2 follow-ups" below for the V1 sort-key tie-break boundary and the cross-shard snapshot non-property.

Caching

Bounded LRU read cache keyed by type_id ‖ primary_id, invalidated by the state machine on Update / Delete. The write path is the source of truth and stays deterministic — the cache is a side index off the committed state, never consulted during apply. Default-on under StateMachine::open; digest-invisible.

Variable-length overflow store

OverflowRef fields hold arbitrary-length bytes without breaking the fixed-width record. The blob travels inside the replicated Create / Update record as a trailer; the state machine splits it out, writes it to a reserved LSM keyspace (type_id = 0xFFFF_FFFF) under a deterministic op-derived handle (op_number << 20) | field_idx, and patches the 8-byte handle into the record's OverflowRef slot. Determinism holds because op_number is assigned by the VSR primary and replicated, so every replica computes the same handle and stores identical bytes. Reads use GetBlob { handle }. Orphaned-blob GC fires precisely at the mutating op: an overflow-field Update frees the superseded blob; a Delete frees the row's blobs.

Equality secondary indexes

ObjectType.indexes lists indexed field ids (replicated catalog). Index entries live in a reserved storage type-slot 0xFFFE0000 | (user_type & 0xFFFF); key = field_id ‖ value_digest8 ‖ pad; value = a per-full-value sorted set of object ids. Keys/bytes are content-derived and id sets sorted, so replicas build a byte-identical index keyspace (digest-covered). CreateIndex backfills via Storage::scan_range over the type's contiguous key range; Create / Update / Delete maintain indexes inline.

Range-and-composite extensions: ObjectType.composite indexes the concatenation of N field values for multi-column equality (SP27); ObjectType.ordered provides sign-correct 8-byte key ordering for range scans via Op::FindRange (SP15).

Built-in constraints

OpResult::Constraint is a deterministic op result.

• NOT NULL derives from Field.nullable and is checked against the codec null-bitmap, but only for well-formed codec records (len == record_size and field_count == #fields) — raw / opaque writers opt out by construction.
• UNIQUE (ObjectType.unique, always ⊆ indexes) consults the equality-index bucket on every Create / Update, excluding self. Op::AddUnique builds the backing index if needed, validates that current data has no duplicate (rejecting without half-applying), then records the constraint.
• FOREIGN KEY (ObjectType.fks): on Create / Update each FK field's value (padded to a 16-byte id) must resolve via storage.get(make_key(ref_type, id)). Read-only against committed state ⇒ deterministic. ON DELETE: RESTRICT, CASCADE (budget-bounded transitive closure), and SET NULL (atomic with the delete). ON UPDATE is inapplicable by model — KesselDB row ids are immutable.
• CHECK programs (ObjectType.checks) run on the deterministic expression VM (kessel-expr) — pure, gas-bounded, terminating.

Query planner

Op::Query takes a conjunction of Pred { field_id, op∈{Eq,Ge,Le}, value }. The planner fetches and intersects the id-sets of all indexed equality predicates; if any exist, it verifies every predicate on just those candidate rows, else it does a filtered scan_range over the type's key range. cmp_field compares per kind (numeric for ints / bool / timestamp, sign-extended for signed / Fixed, lexicographic for byte kinds). Query is read-only and a pure function of committed state, so it is not logged and is trivially identical across replicas.

SP32 / SP62 extend the planner to SQL SELECT * WHERE with indexed fast paths; SP63 wires composite-index narrowing for queries whose equality predicates cover an indexed tuple exactly.

Atomic transactions

Storage has a transaction overlay: begin_txn buffers writes in-memory (reads see them — read-your-writes), commit_txn flushes the whole batch to the WAL with a single fsync then makes it visible, abort_txn drops the overlay (nothing reached WAL / memtable ⇒ nothing to undo). Op::Txn runs its inner data ops through the normal apply path so constraints / indexes / triggers / overflow all compose and roll back together; the read cache is cleared on abort. A transaction is one replicated op, so the serial state machine makes it serializable and replica-identical. DDL / nested txns are rejected.

SQL BEGIN / COMMIT / ROLLBACK (SP55) buffers statements connection-side and emits one Op::Txn at COMMIT.

Honest perf boundary (SP-Bench-Suite T3, 2026-05-28). Op::Txn goes through StateMachine::apply() and takes the write lock for the whole transaction — even when every inner op is read-only. The Perf-A T2 parallel-read bypass (read_only_op dispatch) is GetById-only and does NOT compose with Op::Txn. Under sysbench OLTP read-only this surfaces as a regression N=1 → N=8 (1,241 → 641 tx/s) because N workers serialize on the apply lock instead of running their 10-SELECT brackets in parallel. The KesselDB win on sysbench OLTP write-only is the symmetric story — apply-path is fast at the inner-op level (53K tx/s at N=8, 5.2× Postgres). Closing the RO/RW gap is the named follow-up arc SP-Perf-A-SHARD (sharded apply queues + per-shard read pools, OR routing read-only Op::Txn through the read-pool bypass when every inner op is statically detectable as read-only). See docs/BENCHMARKS.md §3c–§3e for the full transaction-bracket table.

Storage + MVCC

LSM key layout has two shapes:

• Legacy 20-byte keys: type_id(4) ‖ primary_id(16). Used for catalog, indexes, overflow, and internal reserved type-slots.
• MVCC 28-byte keys: type_id(4) ‖ object_id(16) ‖ inverted_commit_opnum(8 BE). Used for every user-type data row. The inverted op_number puts the newest version first under scan_range.

WAL frame: (op_number, kind, type_id, payload, crc32c). A type is a contiguous key range (sets up range scans).

data_row_dispatch(key) at the storage layer routes 20-byte user-type data-row keys (type_id in (0, 0xFF00_0000)) through MVCC primitives at u64::MAX snapshot (reads) and op_number commit (writes). The dispatch is a one-helper-function + 4-call-site change in Storage::{get, put, delete, scan_range} covering ~25-35 data-row I/O sites silently. Replicas reach byte-identical state at every committed log position (3-replica byte-identity tests gate this).

Read-fast-path zero-memcpy (SP-Perf-A T7, 2026-05-29). The memtable + SSTable cached blocks + transaction overlay all store values as Arc<[u8]> rather than Vec<u8>; Storage::get returns the Arc clone directly so the engine's read_only_op bypass walks the byte slice without a heap copy. Combined with T2's parallel-read dispatch (Arc<RwLock<StateMachine>> reader bypass), this lifts point-read throughput to ~4.75M ops/sec at N=16 cores on the vulcan reference server with p50 < 1 µs. The honest ceiling at ~5M ops/sec is the RwLock<StateMachine> reader CAS ping-pong; the named follow-up SP-Perf-A-SHARD sharded apply queues + per-shard read pools is what unlocks the next order of magnitude.

Isolation:

• Snapshot reads via Tx::begin
• Snapshot Isolation write-side via Tx::begin_rw (SP112)
• Cahill serializable SSI via Tx::begin_ssi (SP113 — write-skew impossible by construction)

GC: Op::AdvanceWatermark is a deterministic op in the apply path (SP114); a heartbeat closure (SP115) submits it. The whole stack is mechanically verified across 7 layered TLA+ modules.

Deterministic WASM UDFs

kessel-wasm is a from-scratch zero-dep WASM-MVP-subset interpreter for CHECK constraints and triggers.

Supported instruction surface: i32 / i64 / f32 / f64 values + arithmetic

• comparison + control flow + locals + in-module call + linear memory (load / store / size / grow) + tables + call_indirect with runtime type_idx equality check + bit-manipulation (clz / ctz / popcnt) + sign-extension + canonical NaN (0x7FC0_0000 / 0x7FF8_0000_0000_0000) per WASM determinism rules.

Gas-bounded at 1 unit per executed instruction; trap WasmError::OutOfGas on limit. A bounds-checked decoder + opcode allow-list distinguishes "valid WASM-MVP unsupported" from "invalid garbage". A UDF is part of the replicated catalog; every replica runs byte-identical logic; UDF behavior is replayable from the log. 113+ hand-derived KATs against the official WASM-MVP spec.

Wire protocol gateways

KesselDB exposes the same Op apply path through four wire surfaces. The binary protocol is the default + the deterministic fast path; every other listener is opt-in via a cargo feature, runs on a sibling TCP socket, and is byte-untouched by the binary protocol. See docs/USAGE.md §9 (PostgreSQL) and §10 (HTTP + WebSocket) for operator-side configuration; this section covers the engine-side plumbing.

Each listener has its own max_conns cap (per-listener, not joint — so a saturated HTTP gateway can never starve the binary protocol). The shared engine max_inflight cap bounds total in-flight ops across all listeners honestly.

Binary protocol

The deterministic hot path on the primary port. Length-prefixed Op::encode payloads framed by a 1-byte kind tag; replies are OpResult::encode. This is what the SP69 pipelined-batch perf number measures and what every replication / VSR / Jepsen oracle exercises. Bearer-token authed via the 0xFC handshake frame (SP43).

HTTP gateway (--features http-gateway)

Translates HTTP/1.1 requests into the same engine apply path via the kessel_http_gateway::EngineApply trait that EngineHandle impls. Routes: /v1/sql, /v1/op, /v1/health, /v1/metrics, /v1/ws. Bearer auth shared with the binary protocol. HTTPS variant runs on a third listener via the existing rustls config used by the binary listener (with the tls feature).

The gateway crate kessel-http-gateway has zero external (non-workspace) runtime dependencies. Default cargo build -p kesseldb-server (without --features http-gateway) does not link the gateway crate — cargo tree verifies the binary stays untouched.

WebSocket arm (under --features http-gateway)

The WebSocket arm of the HTTP gateway exposes a long-lived /v1/ws upgrade that frames raw Op::encode() payloads under the kessel-op-v1 subprotocol. RFC 6455 strict handshake + binary frames only + bounded send queue (16 messages) + 30s ping/pong heartbeat. There is no separate ws-gateway feature — the WebSocket session model lives inside kessel-http-gateway alongside the HTTP routes; the crate, the Bearer auth surface, and the EngineApply trait are shared.

PostgreSQL wire (--features pg-gateway)

Speaks the PostgreSQL Frontend/Backend Protocol v3.0 — the same wire that libpq / psql / pgcli / JDBC / psycopg / pgx / tokio-postgres / sqlx-pg all speak. Per-connection std::thread. Connection cap defaults to DEFAULT_MAX_PG_CONNS = 256; the PG and HTTP caps are independent.

V1 scope:

• Simple Query (Q message): single statement per Q, multi-statement rejected with 42601. Streaming RowDescription → DataRow* → CommandComplete → ReadyForQuery per query.
• Extended Query (SP-PG-EXTQ V1, 2026-05-29) — full V1 message set P (Parse) / B (Bind) / D (Describe) / E (Execute) / S (Sync) / C (Close) / H (Flush). Per-connection SessionState holds named + unnamed prepared statements + portals up to MAX_PREPARED_STATEMENTS_PER_CONN = MAX_PORTALS_PER_CONN = 4096. Parse stores SQL VERBATIM (no parse, no AST cache — SQL parse errors surface at Execute time so the engine catalog state governs the message). Bind validates parameter format codes (V1 rejects binary with 0A000 — V2 SP-PG-EXTQ-BIN), enforces parameter count vs Parse's OID hints (mismatch → 08P02), and stores text-format parameter values into the portal. Describe 'S' emits ParameterDescription + RowDescription/NoData; Describe 'P' emits RowDescription/NoData (parameters frozen at Bind time per PG §55.2.3). Execute substitutes $N text-format parameters into the SQL and dispatches through EngineApply::apply_sql; max_rows > 0 emits PortalSuspended with buffered cursor state so a re-Execute resumes pagination. Sync emits ReadyForQuery('I'), clears the per-connection error_state (set on any prior dispatch error), and drops the unnamed portal. Close drops the named statement or portal; CloseComplete emitted on success even for missing-name no-ops per PG §55.2.3. Flush triggers an outbound stream flush (no bytes, no state change). End-to-end verification: a real psycopg2.connect(...) + cur.execute("SELECT * FROM pgtest WHERE id = %s", (42,)) returns real rows on vulcan (SP-PG-EXTQ T5 / commit cec17c4). Full ORM-suite smoke against SQLAlchemy + JDBC + Drizzle + Prisma is post-V1.1 (SP-PG-EXTQ T8 / T11 / T12 — still OPEN at the time of writing).
• SCRAM-SHA-256 auth (RFC 5802 + RFC 7677, 4096 iterations) via the Bearer ↔ SCRAM bridge: the operator's ServerConfig.token IS the SCRAM password input. One credential surface; rotating the Bearer token rotates both HTTP-Bearer and PG-SCRAM atomically.
• Type-OID mapping for KesselDB FieldKind → PG type catalog (Bool, int2/4/8, text, bytea, timestamptz, numeric). Text-format wire encoding only — binary-format wire is V2 SP-PG-EXTQ-BIN.
• Cap-overflow rejection as wire-level ErrorResponse('S=FATAL', 'C=53300', 'M=sorry, too many clients already') emitted before the close, so libpq surfaces the structured rejection.
• Idle timeout (pg_idle_timeout, default 600s) emits FATAL 57014 terminating connection due to idle timeout before close.
• OpResult → SQLSTATE mapping: Exists → 23505, Unauthorized → FATAL 28000, Unavailable → FATAL 57P03, SchemaError(msg) → 42P01 / 42703 / 42804 / 42601 / 42000 (string-match heuristic), Constraint → 23502 / 23505 / 23503 / 23514 / 23000, TxAborted → 40001 / 25006 / 58030.
• Scatter-scan transparency: PG-wire dispatches every SQL through EngineApply::apply_sql; the underlying engine routes scan-shaped ops via Route::Scatter and merges per-shard OpResult::Got slots. PG-wire is byte-identical between K=1 and K=N.

pg_catalog stubs (SP-PG-CAT)

GUI tools (pgAdmin / DBeaver / DataGrip / Metabase / Tableau / Looker) issue 5-50 introspection queries against pg_catalog.* and information_schema.* to populate their UI tree on connect. SP-PG V1 returned 42P01 undefined_table for every such query, so GUI tools refused to display the connection. SP-PG-CAT closes that boundary by intercepting the query at the dispatch layer (kessel_pg_gateway::pg_catalog::catalog_query_hook) BEFORE the engine apply path and synthesizing a wire-coherent response from the live KesselDB catalog.

Synthesized catalog tables (PG-canonical column shapes locked vs the upstream src/include/catalog/pg_*.dat + pg_*.h files):

• pg_namespace — 3 canned schemas (pg_catalog, public, information_schema)
• pg_class — one row per KesselDB user table
• pg_attribute — one row per (table × column) with the V1 type-OID map
• pg_type — 13 canned rows for the OIDs V1 actually emits
• pg_index — one row per KesselDB index
• pg_constraint — one row per UNIQUE / FK / CHECK
• information_schema.tables / .columns / .schemata / .key_column_usage / .table_constraints — the SQL-standard catalog mirror with SQL-standard type names (preferred by Metabase / Tableau / Looker / dbt over pg_catalog)
• information_schema.views / .routines — well-framed empty

SQL helper functions: version(), current_database(), current_schema(), current_user, session_user, pg_table_is_visible(oid), pg_get_userbyid(oid), format_type(oid, typmod), current_setting('<guc>'), SHOW <guc>.

Indexes + constraints round-trip through admin frames LIST_INDEXES_TAG=0xF5 and LIST_CONSTRAINTS_TAG=0xF4 that read StateMachine::catalog() engine-thread-local with no SM mutation (mirrors the existing DESCRIBE_BY_NAME_TAG=0xF7 / LIST_TABLES_TAG=0xF6 admin pattern). The intercept is purely additive: every SP-PG V1 KAT continues to pass because the hook returns None for non-pg_catalog SQL and the existing engine.apply_sql path runs unchanged.

The gateway crate kessel-pg-gateway has zero external (non-workspace) runtime dependencies. Default cargo build -p kesseldb-server (without --features pg-gateway) does not link the gateway crate.

Mechanically-checked rigor artifacts (S1, S3)

kesseldb-tla/ — seven layered TLA+ modules with TLC baselines:

Every module preserves prior invariants; the SP109–SP116 discipline is "never weaken a test" — refinements tighten or restate.

Jepsen-style multi-replica linearizability (S3, SP117) — 5 hand-derived Jepsen tests in kessel-vsr::sim::tests validate that the SP116 storage-layer transparent MVCC dispatch preserves linearizability across the full VSR + MVCC stack under partition + message loss. Cluster::drive_until_digests_converge extends the simulation past replies-complete so isolated minority replicas finish state-transfer and catch up.

Limitations & v2 follow-ups

Consolidated list of named deferrals across the codebase. Each is a deliberate boundary, not a hidden gap.

Cross-shard reads (SP-A)

• Sort-key tie-break by (value, shard_id), not (value, object_id). Deterministic + reproducible for fixed K, but cross-K with sort-value ties may order tied rows differently. Per-shard records don't carry the object_id (it's the storage key, not the record), so an oid-based tie-break would require a new Op::SelectSortedWithKey. Deferred until a workload needs it; the 85-seed K-invariance sweep confirmed this is acceptable for V1 because unique sort values are the common case.
• Cross-shard snapshot is not consistent: a scatter read can see rows committed on shard A at opnum_a=100 and rows on shard B at opnum_b=200 (independent counters). Cross-shard consistent snapshot is an explicit non-goal.
• ShardCaller::call cannot be interrupted mid-reply (std::net::TcpStream has no cancellable read); per-shard read_timeout (default 30s) is the upper bound on cancel latency.
• Hard-fail by default: a single per-shard non-Got slot poisons the merged result. Opt-in best-effort mode via ScatterContext::partial_on_timeout lets callers receive the surviving shards' rows plus a Vec<u32> of failed shard ids.

Out of arc (each its own future arc): SP-B aggregate combine, SP-C streamed sorted merge over indexes, SP-D GroupAggregate cross-shard combine, SP-E SQL-text routing. Cross-shard Join remains an explicit non-goal.

FindBy / FindByComposite extend SP-A via the OidConcat ScatterKind — the per-shard secondary index doesn't carry rows from other shards, so the router fans out an indexed-equality lookup to every shard and unions the resulting oid sets.

PostgreSQL wire

V2 follow-ups (each its own arc). Extended Query has SHIPPED at V1.1 (SP-PG-EXTQ); it is no longer on this list. What remains:

• Binary-format wire encoding (per-column negotiated in Bind) — SP-PG-EXTQ-BIN
• RETURNING clause
• CancelRequest action (V1 generates BackendKeyData but takes no action)
• GUC plumbing for SET timezone etc.
• COPY FROM STDIN / COPY TO STDOUT — SP-PG-COPY
• TLS via SSLRequest 'S' reply + rustls (V1 plaintext only)
• MD5 auth fallback for legacy clients (PG 14+ deprecated)
• SCRAM channel binding (SCRAM-SHA-256-PLUS)

See SP-PG design spec §2.2 for the full deferred list.

pg_catalog stubs

• pg_proc real function listing (SP-PG-CAT-PROC)
• pg_stat_* runtime stats (SP-PG-CAT-STATS)
• Arbitrary pg_catalog SQL via AST walker (SP-PG-CAT-AST) — V1 handles named queries only; ad-hoc JOINs against catalog tables fall through to the engine and error
• psql \d+ extended output
• Multi-database pg_database (blocks on KesselDB multi-database support)
• Per-query catalog cache invalidated on DDL (SP-PG-CAT-CACHE — matters at ≥1000 tables)
• Cross-schema queries (blocks on SP-NS)

Parquet

• 4+ deep nesting (List<List<List<List<T>>>> etc.) rejects with a typed Unsupported error awaiting a real pyarrow fixture that exercises that depth — synthetic tests don't justify the classifier extension
• Legacy LZ4 framing (codec id 5) — pyarrow ≤ 8 default; modern pyarrow uses LZ4_RAW (codec id 7) which IS supported

Storage / SQL

• ON DELETE SET DEFAULT — needs per-column defaults first
• ON UPDATE referential actions — inapplicable by model (row ids are immutable; the trigger has no condition under which it could ever fire)
• ALTER TABLE DROP / ALTER COLUMN, DROP INDEX — only ADD COLUMN and DROP TABLE are wired
• Auto-id / sequences — callers supply the 16-byte object id today
• Range / composite index narrowing for > / < — equality predicates narrow via the planner; range predicates still verify via the expression VM

Operations

• TLS for the binary protocol is implemented (rustls); TLS for HTTP and PG wire is V2
• No incremental backup / PITR — Op::Snapshot produces a flat crash-consistent copy
• No per-table or role-based authz beyond shared Bearer token / SCRAM password

The determinism seam

Everything above storage in KesselDB is a pure function over an injected clock, disk, and network (kessel-io). Production injects real I/O; kessel-sim injects a seeded, fault-injecting fake. The whole database runs deterministically from one u64 seed — this is what makes a from-scratch VSR reimplementation verifiable rather than hopeful.

kessel-sm, kessel-catalog, and kessel-codec contain zero I/O, clock, or RNG calls.

Full text: Architecture → The determinism seam.

MVCC & VSR

KesselDB ships Viewstamped Replication as its replication protocol (primary assigns op-number + deterministic timestamp; Prepare → f+1 PrepareOk → Commit; backups apply in op-number order; view-change on primary timeout; client table for exactly-once retried client batches). Fixed cluster size (3 or 5); membership reconfiguration is out of scope for Sub-project 1.

The MVCC keyspace is a 28-byte type_id(4) ‖ object_id(16) ‖ inverted_commit_opnum(8 BE) layout living in the same LSM as the 20-byte legacy keyspace; the inverted op_number puts the newest version first under scan_range. The data_row_dispatch(key) discriminator at the storage layer routes 20-byte user-type data-row keys through MVCC primitives at u64::MAX snapshot (reads) and op_number commit (writes) — no apply-arm rewrites needed.

Isolation: snapshot reads, SI write-side, Cahill serializable SSI (write-skew impossible by construction). GC: Op::AdvanceWatermark is a deterministic op in the apply path. The whole stack is mechanically verified by TLC across 7 layered TLA+ modules (kesseldb-tla/MVCC*.tla + Replication.tla).

Full reference: Architecture → Replication (VSR) and Architecture → MVCC.

Mechanically-checked rigor artifacts: kesseldb-tla/ (Replication.tla TLC: 528M distinct states / depth 21 / 0 violations).

Sharding & scatter scan

A deployment runs K independent VSR shard groups behind a router with a rendezvous key→shard mapping. A single-shard transaction stays on its shard's own VSR group (serializable, fast path). Cross-shard Op::Txn is deterministic (Calvin-style) — slices durably totally ordered by a sequencer group, then each shard applies its slice in that order via a decide → commit — no 2PC, no coordinator-failure hole.

Cross-shard reads (SP-A) — Select / QueryRows / SelectFields / SelectSorted automatically scatter across every shard via scatter_scan. Unordered scatter = shard-id-deterministic concatenation. Sorted scatter = BinaryHeap k-way merge of already-sorted per-shard streams. K-invariance is locked across K ∈ {1, 2, 4, 8, 16} by an 85-seed property sweep — with unique sort values, merged output is byte-identical to the K=1 baseline.

Full reference: Architecture → Sharding & cross-shard transactions and Architecture → Cross-shard reads (SP-A).

Wire protocols

KesselDB exposes the same Op apply path through four wire surfaces:

• Binary on the primary port — deterministic fast path, default cargo build, no external deps; length-prefixed [u32 LE len][payload] frames.
• HTTP/1.1 with --features http-gateway — /v1/sql, /v1/op, /v1/health, /v1/metrics. JSON responses, Prometheus metrics.
• WebSocket with --features http-gateway (same crate, same feature) — /v1/ws upgrade, kessel-op-v1 subprotocol, binary frames carrying Op::encode().
• PostgreSQL Frontend/Backend v3.0 with --features pg-gateway — Simple Query + SCRAM-SHA-256 + Bearer↔SCRAM bridge; pg_catalog + information_schema stubs (SP-PG-CAT) so pgAdmin, DBeaver, DataGrip, Metabase, Tableau connect + browse out of the box.

Per-listener max_conns caps mean a saturated gateway can never starve the binary protocol. The shared engine max_inflight bounds total in-flight ops across listeners.

Full reference: Architecture → Wire protocol gateways. Wire-shape constants and admin-frame tags: Wire protocol.

Wire protocol

Each message on the binary wire is length-prefixed: [u32 little-endian length][payload].

This is intentionally tiny — any language can speak it with a socket and the length framing. The kessel-client crate implements all of it; clients/python/kesseldb.py is a stdlib-only Python reference.

Full reference: Usage guide (full) §12.

Performance

Full methodology, measured numbers, scaling model, and cloud projections:

KesselDB performance

Honest numbers, the model behind them, and order-of-magnitude projections for common cloud configurations.

What is measured vs projected. The tables under Measured are real runs on two reference machines (described generically below). The Cloud projections table is extrapolated from the measured single-core throughput plus the storage/network model — it is not measured on those instances. Treat it as planning guidance, not a benchmark result.

Reference machines

• Reference server — a 16‑core x86‑64 Linux box (≈3 GHz class), local NVMe SSD, loopback networking. Shared/old disk near capacity, so durable numbers are conservative.
• Reference laptop — an x86‑64 Windows 11 developer laptop.

No tuning, default build (zero external dependencies), single deterministic writer thread.

The model (why the projections are what they are)

KesselDB is a single deterministic state machine. That fixes how it scales, and the projections fall straight out of it:

1. Steady-state op throughput is single-core-bound. One writer applies operations in order. Throughput tracks per-core clock × IPC, not vCPU count. More cores buy connection concurrency and read parallelism, not a higher write rate.
2. Durable throughput is fsync-latency-bound. Server-side group commit amortises one fsync over the whole in-flight batch, so effective durable rate ≈ batch size ÷ fsync latency. Fast local NVMe (~50–200 µs) → tens of thousands/s; network-attached volumes (~0.5–2 ms) → still thousands–tens of thousands/s because the batch grows under load.
3. Latency is round-trip-bound. TCP_NODELAY removes the Nagle/delayed-ACK stall (a ~40 ms/round-trip cliff on Linux); pipelining and group commit amortise the remaining RTT.
4. Indexed and columnar reads are sub-linear and CPU/cache-bound, so they track single-core performance and are largely independent of table size.
5. Sharding scales single-shard work horizontally. Point ops and single-shard transactions go straight to the owning shard group, so aggregate throughput grows ~linearly with shard count. A cross-shard transaction additionally pays one sequencer round-trip plus a decide+commit round-trip per participating shard (deterministic, no locks/2PC blocking); the router currently serializes cross-shard commits to drive the global order, so they are the deliberate slow path — keep transactions single-shard where latency-critical.

Measured

Single connection / single thread unless noted.

The columnar fast-path is also ~1,800× on the reference laptop (~14 µs vs ~23 ms) — the absolute µs differs with single-core speed; the shape (sub-linear, scan eliminated) does not.

Every figure is reproducible from the test suite / kessel-bench, and each query accelerator is guarded by a randomized equivalence oracle (the accelerated result is proven identical to the brute-force scan).

Cloud projections (extrapolated — not measured)

Applying the model to representative instance families. Single-core class is the dominant factor for CPU-bound paths; storage class for durable writes. Projection, not a benchmark.

Reading the table:

• In-mem / compile / read paths scale with single-core speed and are roughly cloud-instance-independent beyond clock/IPC — pick the fastest single core, not the most vCPUs.
• Durable writes depend almost entirely on storage. Network SSD has higher per-fsync latency, but group commit makes the durable rate climb with offered concurrency, so a busy server still reaches tens of thousands/s. For the lowest write latency, prefer local NVMe (instance store) and accept its ephemerality, or replicate.
• Columnar / indexed reads (MIN/MAX via the order index, range/band narrowing, prepared-statement cache) are sub-linear and table-size-independent — the projections there are about absolute µs, not whether the optimisation applies.

Reproducing

cargo test --workspace --release        # functional + equivalence oracles
cargo run -p kessel-bench --release -- --help

Numbers move with hardware; the relationships in the model (single-core-bound ops, fsync-bound durability, sub-linear indexed reads, scan-free MIN/MAX) hold across platforms.

Thesis

The five thesis pillars (deterministic / verifiable / replayable / zero-dep / honest-docs) and the strategic-tier backlog S1–S4 (all shipped):

KesselDB — Project Thesis

Date adopted: 2026-05-19
Status: adopted
Source: 2026-05-19 strategic review; recorded in memory/project_kesseldb_strategic_tier.md and the strategic-tier context section of docs/superpowers/specs/2026-05-19-kesseldb-subproject108-int96-decimal.md.

The thesis

Deterministic replicated SQL with verifiable behavior and replayability.

Each term has a concrete meaning in this codebase:

• Deterministic replicated SQL. The state machine (kessel-sm) and the Viewstamped Replication layer (kessel-vsr) together guarantee that, given the same log prefix, every replica produces byte-identical committed state. No wall-clock reads, no thread-scheduling-dependent ordering, and no implicit allocator-dependent behavior appear inside the deterministic path. The SQL surface (kessel-sql, kessel-expr) is served from that same deterministic core; a SQL query is a function of committed log state, nothing else.

• Verifiable behavior. "Looks rigorous" is insufficient. The project ships mechanically-checked artifacts (S1: TLA+/model-checked safety specs for the replication protocol and MVCC) and externally-attested rigor (S3: Jepsen results against a real 3-node cluster under partition + clock skew + process kill). Internal tests are necessary but not sufficient; the thesis demands artifacts an outsider can check independently.

• Replayability. Every committed behavior is a function of a seed corpus and an ordered log. The kessel-sim fault simulator has run the historically difficult seed 7 since M3; the seeded adversarial-replay pattern is the debugging discipline: any bug report reduces to a (seed, log) tuple. Debugging IS replay. The strategic-tier WASM UDF work (S4) extends this property to user-defined extensions: a WASM UDF is sandboxed, gas-accounted, and deterministic, so a UDF's behavior is also replayable.

The strategic-tier backlog items S1–S4 (listed below) turn these three properties from design intentions into mechanically-provable and externally-attested artifacts.

Comparison to the great database theses

Each great database system has a deep core idea that makes it distinctive. KesselDB's thesis is a peer of these:

This framing is the user's verbatim rationale from the 2026-05-19 strategic review: "The great database systems usually have a deep core idea. KesselDB's path to incredible is probably: deterministic replicated SQL with verifiable behavior and replayability — that's the most differentiated part of the design."

What this thesis commits to

These are design rules that flow directly from the thesis. They are not aspirational; they are constraints that every slice must satisfy.

Deterministic kernel

• No wall-clock reads, no thread-nondeterminism, no implicit allocator-dependent ordering inside kessel-sm, kessel-catalog, kessel-codec, kessel-vsr, or kessel-expr.
• Replication produces bytewise-identical committed state across all replicas given the same log prefix. The seeded VSR simulation corpus (100 seeds × 2 runs identical; seed 7 green at every merge) is the ongoing gate.
• The kessel-io seam is the single injection point for clock, disk, and network. Production injects real I/O; kessel-sim injects a seeded, fault-injecting fake. Nothing above kessel-io may cross this boundary without going through the seam.

Verifiable behavior

• The project ships mechanically-checked safety invariants (S1: TLA+/TLC or Apalache model-checked specs for linearizability, exactly-once apply, and log-prefix safety under partition + restart).
• The project ships externally-attested rigor (S3: Jepsen linearizability checker — Knossos/Elle — against a real 3-node cluster).
• Gate figures in slice records are real measured numbers, not estimates. Every plan deviation is disclosed in the slice's permanent record (the SP107 V1-ordering-defect disclosure, the SP108 plan-arithmetic correction, and the T4 cross-physical-type-pin gate-caught correction are the discipline, not the exception).

Replayability

• Every commit's behavior is a function of its seed corpus and log state. This is not a goal; it is enforced by the determinism seam.
• Bug reports reduce to (seed, log) tuples. Debugging IS replay: reproduce the simulation run, replay the log, observe the defect.
• The kessel-sim seed-7 liveness test (the historically-difficult seed) runs on every CI merge and is a hard gate.

Zero-dependency kernel

• External dependencies are kept out of the deterministic path. The kessel-parquet crate (pure-Rust, zero external deps, #![forbid(unsafe_code)]) and the hand-rolled zero-dep HMAC-SHA256 / RFC-1952 / Snappy / Thrift implementations are existing examples of this discipline.
• New external dependencies in the default build require an explicit thesis-fit justification. Feature-gated dependencies (rustls, objstore) are acceptable because they do not enter the deterministic core path.

Honest-engineering documentation

• Every slice's permanent record in docs/superpowers/specs/ names its plan deviations, gate disclosures, and any retroactive corrections.
• The honest-gate accounting pattern (e.g., "Honest gate: 425→484 (+59; not zero-delta") is mandatory, not optional. Suppressing gate disclosures is a thesis violation.
• Going forward, every spec gets a one-line thesis-fit note (see the per-slice rule below).

What this thesis explicitly does NOT commit to

The thesis is only useful if it has boundaries. The following are explicitly out of scope:

Not optimizing for:

• Vendor-locked feature parity with PostgreSQL or MySQL. Coverage breadth is not a thesis-defining goal; coverage of features that prove deterministic replicated correctness is.
• Cost-based optimizer feature parity with DuckDB or Snowflake. A CBO is deferred until SQL workload demand justifies it and until the thesis core is complete.
• Storage-format breadth beyond what proves the thesis. Parquet is supported because the Iceberg/lakehouse path (OBJ-3) is on the thesis trajectory. Arbitrary new formats are deferred until the thesis-fit justifies them.
• SaaS-style operational features that bloat the binary (e.g., cloud-native autoscaling APIs, multi-cloud storage tiering, managed backup).

Not a replacement for:

• PostgreSQL. KesselDB is not Postgres-with-replication. Users who need PostgreSQL's extension ecosystem, maturity, or operational tooling should use PostgreSQL.
• Streaming databases (Materialize, RisingWave). Their thesis is continuous incremental view maintenance over event streams; KesselDB's thesis is deterministic replicated correctness over a committed log.
• Graph databases. No graph traversal primitives are on the roadmap.
• Feature stores. Online/offline feature store semantics are not a goal.

Strategic-tier backlog (S1–S4) — all four SHIPPED (2026-05-24)

Status addendum (2026-05-24): All four strategic-tier items below shipped over the SP109‑SP118 session arc (with the S2 MVCC sub-arc completing at SP116). The original "backlog" framing below is kept verbatim for historical context; the per-slice records under docs/superpowers/specs/2026-05-2* are the authoritative ship proofs, and the corresponding STATUS rows live in docs/STATUS.md.

Quick map:

• S1 → SP109 (Replication.tla, TLC 528M states / depth 21 / 0 violations)
• S2 → SP110-SP116 (7-layer MVCC TLA+ stack: MVCCStorage → MVCCTx → MVCCSi → MVCCSsi → MVCCGc → MVCCCutover; storage-layer transparent MVCC dispatch via data_row_dispatch(key))
• S3 → SP117 (5 hand-derived Jepsen tests in kessel-vsr::sim::tests
  • new Cluster::drive_until_digests_converge API)
• S4 → SP118 (zero-dep kessel-wasm crate, evolved through SP119‑SP124 to cover i32/i64/f32/f64 + memory + tables/call_indirect
  • canonical NaN)

The "backlog" sections that follow describe these items as they were originally framed during the 2026-05-19 strategic review. Read them as intent; read STATUS.md / kesseldb-tla/ / kessel-wasm/ for the shipped artifacts.

These four items were added during the 2026-05-19 strategic review. They are ordered by thesis-leverage: each converts an existing design property from "intended" to "provable" or "externally attested." S1 is the immediate next slice after THESIS.md.

Sources: memory/project_kesseldb_strategic_tier.md and the strategic-tier context section of docs/superpowers/specs/2026-05-19-kesseldb-subproject108-int96-decimal.md.

S1 — TLA+/model-checked safety specs

Thesis lever: verifiable behavior (the single artifact that converts "looks rigorous" → "is provably so").

Safety invariants for the replication log, checked mechanically via TLC or Apalache: linearizability, exactly-once apply, log-prefix safety under partition + restart. The kesseldb-tla/ directory. Starts immediately after this THESIS.md commit. This is the TigerBeetle rigor lever: TigerBeetle's formal protocol verification is a primary reason the system is trusted in financial contexts. KesselDB's deterministic log is the right substrate for the same discipline.

S2 — Serializable MVCC / Snapshot Isolation over the deterministic log

Thesis lever: deterministic + replayable (proves "consensus + SQL can be simpler than MVCC-centric systems" concretely).

Snapshot reads without blocking writes; deterministic conflict resolution; long-running reads without stalling compaction; replicated MVCC correctness proofs. Multi-slice (estimated 4–6 slices). MVCC state is part of the log; every snapshot is replayable from the log prefix that precedes it.

S3 — Jepsen harness against a real 3-node cluster

Thesis lever: verifiable behavior (externally-attested; the gold-standard adoption signal for serious distributed databases).

Linearizability checker (Knossos/Elle) under partition + clock-skew + process-kill against a live cluster. Pairs with S1: together they complete the full rigor story — S1 proves the protocol correct by construction, S3 demonstrates correctness under real fault conditions. Published Jepsen results have been the canonical trigger for institutional adoption of distributed databases (CockroachDB, TigerBeetle, etc.).

S4 — Deterministic in-tree WASM UDF runtime

Thesis lever: deterministic + replayable + zero-dep (the distinctive extension story: most databases cannot safely combine extensibility with deterministic replication).

Sandboxed, gas-accounted, zero-import WASM UDF runtime inside the deterministic core. A WASM UDF is part of the replicated catalog; every replica runs byte-identical logic; UDF behavior is replayable from the log. This subsumes the existing open "WASM trigger sandbox" item from SP4/SP8. The gas-accounting constraint prevents non-termination without breaking the determinism property.

Per-slice thesis-fit note (rule, going forward)

Every future spec in docs/superpowers/specs/ must include a one-line thesis-fit note in its decisions section, naming which thesis pillar(s) the slice strengthens:

Example note (in a decisions section):

Thesis fit: verifiable (source-independence pin proves format-agnostic
decode correctness), honest-docs (7th e2e fail-closed + T4 plan-arithmetic
correction disclosed).

Retroactive mapping for recent slices (so future maintainers reading those records can locate the thesis contribution):

Process note: how this thesis was adopted

The 2026-05-19 strategic review was user-led (via a ChatGPT strategic-tier analysis session). The thesis sentence — "deterministic replicated SQL with verifiable behavior and replayability" — was identified by the user as the most differentiated path for KesselDB, by analogy to the core ideas of the great database systems (PostgreSQL, FoundationDB, DuckDB, TigerBeetle, Datomic).

The user then resolved a sequencing question: finish SP108 (OBJ-2c-4 INT96/DECIMAL) first, then write docs/THESIS.md, then start S1 (TLA+ specs). That decision is recorded in memory/project_kesseldb_strategic_tier.md and mirrored in the strategic-tier context section of the SP108 record.

This document is the output of that decision. It is a permanent record, not a living document; if the thesis is refined in a future review, a new dated entry supersedes this one rather than editing it in place.

Agents guide

The machine-first operating guide — build/test/run commands, the CLI contract, the wire protocols, the repo map, and the working rules future agents must follow if they modify this repo:

AGENTS.md — operating guide for KesselDB

The machine-first entry point. Read this first. Humans: see README.md; deep usage is docs/USAGE.md.

What this is

KesselDB: a deterministic, replicated SQL database in pure Rust, zero external dependencies. PostgreSQL-style flexibility (runtime tables, online DDL, SQL, constraints, triggers, transactions) on a TigerBeetle-style core (deterministic state machine, LSM+WAL, Viewstamped Replication, seeded simulation testing).

Status: every named production-readiness gate is met; see docs/STATUS.md for the precise gate table and per-slice history. Every claim is backed by the test suite.

Build / test / run

cargo build --workspace                          # all crates, no external deps, no native steps
cargo test  --workspace                          # 2442 default tests
cargo test  --workspace --features pg-gateway    # 2470 (adds SP-PG + SP-PG-CAT + SP-PG-EXTQ V1 + V2 hardening + SP-PG-COPY V1)
cargo test  --workspace --features pg-gateway,http-gateway,kessel-http-gateway/test-server   # 2503 — full matrix

cargo run --release --bin kesseldb -- 127.0.0.1:7878 ./data   # single open node, binary protocol only

# All wire surfaces (binary + HTTP + WS + PG) on a single node:
cargo build --release --bin kesseldb -p kesseldb-server --features pg-gateway,http-gateway
KESSELDB_TOKEN=secret \
KESSELDB_HTTP_ADDR=127.0.0.1:8080 \
KESSELDB_PG_ADDR=127.0.0.1:5432 \
  ./target/release/kesseldb 127.0.0.1:7878 ./data

Rust stable 1.95+. The test suite is the source of truth — if it is green, the documented behaviour holds.

Talk to it without writing code (preferred for agents)

The kessel CLI is line-oriented with meaningful exit codes — do not scrape text to detect success.

cargo run -q -p kessel-client --bin kessel -- "CREATE TABLE t (v U64 NOT NULL)"
cargo run -q -p kessel-client --bin kessel -- "INSERT INTO t ID 1 (v) VALUES (42)"
cargo run -q -p kessel-client --bin kessel -- "SELECT SUM(v) FROM t"     # => = 42
echo "SELECT * FROM t ID 1" | cargo run -q -p kessel-client --bin kessel  # pipe

kessel [--addr HOST:PORT] [--token TOKEN] ["SQL"]. Exit 0 = success, 1 = statement error / connection failure, 2 = bad usage. With no SQL arg it reads stdin (one statement per line; #/-- lines are comments). SQL reference: docs/USAGE.md §4.

From Rust, use kessel_client::{Client, ClusterClient, format_result}.

Wire protocols (for non-Rust clients)

KesselDB exposes four wire surfaces; all run on the same engine and apply the same Op. The binary protocol is the deterministic fast path and the default.

• Binary — length-prefixed [u32 LE len][payload]. First payload byte selects mode: plain Op::encode(), 0xFE+SQL, 0xFD+session frame (exactly-once), 0xFC+token (auth), 0xFB (stats), 0xFA+dir (snapshot). Full table in docs/USAGE.md §12.
• HTTP/1.1 — /v1/sql, /v1/op, /v1/health, /v1/metrics. JSON responses. --features http-gateway. See docs/USAGE.md §10.
• WebSocket — /v1/ws upgrade, kessel-op-v1 subprotocol, binary frames carrying Op::encode() payloads. Same http-gateway feature. See docs/USAGE.md §10 → WebSocket.
• PostgreSQL Frontend/Backend v3.0 — Simple Query + Extended Query (Parse/Bind/Describe/Execute/Sync/Close/Flush, binary params + binary results) + SCRAM-SHA-256 + Bearer↔SCRAM bridge + COPY FROM/TO STDIN (text + CSV + binary formats). Real-driver verified on vulcan: psycopg2 ✓ SQLAlchemy 2.0 ✓ psycopg3 ✓ asyncpg ✓ pgJDBC 42.7.4 ✓ (both simple AND extended modes). --features pg-gateway. See docs/USAGE.md §9.

Repo map

Working rules (apply if you modify this repo)

1. Test-driven, one slice at a time. Add/extend a test, implement, keep cargo test --workspace fully green, then commit. Never commit red or unverified code.
2. Claims never exceed tests. Docs (README, STATUS, specs) state only what the suite proves. If a benchmark contradicts an expected result, report the real number and reframe — do not overclaim (see the SP46 / SP48 self-corrections for the expected discipline).
3. One spec per slice under docs/superpowers/specs/, and update docs/STATUS.md (gate/table) + README.md test count.
4. Zero external dependencies is a hard design rule. Don't add crates.
5. Determinism is sacred. Anything affecting replicated state must be deterministic; engine-local accelerators (caches, blooms) must be digest-invisible and proven so by the full corpus.
6. Commit per slice (history has shown disk-full truncations; per-slice commits are the recovery mechanism). End commit messages with the project's Co-Authored-By line.

Status

Current capabilities summary, production-readiness gate, and the per-slice historical narrative — every claim is backed by the test suite:

KesselDB — Status

Honest milestone tracker. Updated every milestone. "Done" means code + tests committed and passing.

Current capabilities (2026-06-02)

What a node running on today's main actually does. Every line below is covered by the workspace test suite (2442 default / 2470 with --features pg-gateway / 2503 with all gateway features — vulcan-measured 2026-06-02 at HEAD f2a18e5, fresh full sweep; the prior 2063 / 2074 / 2078 figures were delta-derived from an earlier base measurement and had drifted from the actual workspace count).

Coherent state of the union (2026-06-02):

• Non-correlated WHERE subqueries (SP-PG-SQL-SUBQUERY-WHERE, 2026-06-04). SELECT name FROM users WHERE id IN (SELECT user_id FROM orders WHERE total > 100), the NOT IN complement, and the scalar form WHERE price = (SELECT MAX(price) FROM products) (= <> != < <= > >=, inner one-row/one-column) all work over the PG wire. Two-phase at the gateway: a quote-skipping, paren-balancing scan detects <IN|NOT IN|cmp> (SELECT …); the inner SELECT runs FIRST through the normal render path (so aggregates / WHERE inside the inner work for free), its single column's values are spliced into the outer query as a literal list / scalar (typed from the inner RowDescription — ints bare, text single-quoted + escaped), and the rewritten outer re-dispatches normally. NO Op/wire/storage change → determinism oracles byte-untouched. Empty inner: IN (∅) → 0 rows, NOT IN (∅) → all non-NULL rows. Inner ≠ 1 column (42601) / scalar > 1 row (21000) error cleanly. NON-correlated, one-subquery-per-WHERE V1; correlated / EXISTS / FROM-subquery / SELECT-list / multiple subqueries are named follow-ups. New psql smoke scripts/sppgsqlsubquerywhere-smoke.py (10/10 psycopg2 stages on vulcan).
• SELECT DISTINCT row deduplication (SP-PG-SQL-DISTINCT, 2026-06-04). SELECT DISTINCT region FROM t (unique column values), SELECT DISTINCT a, b FROM t (unique tuples), and SELECT DISTINCT * FROM t (unique whole rows) dedup result rows over the PG wire; composes with WHERE and ORDER BY (sorted scan order preserved post-dedup). NULL is NOT distinct from NULL. The SELECT N tag reports the DEDUPED count. RENDER-LAYER arc: SELECT DISTINCT … compiles to the SAME Op as the non-distinct form (engine returns all rows), and the gateway dedups the emitted DataRows by their exact projected cell tuple (first occurrence in scan order) — NO Op/wire/storage change, so the determinism oracles are byte-untouched. Non-distinct SELECTs stay byte-identical. DISTINCT ON (…), DISTINCT over JOIN, and DISTINCT over aggregate/GROUP BY are NAMED FOLLOW-UPS — cleanly errored, never returned with duplicates. New psql smoke scripts/sppgsqldistinct-smoke.py (7/7 psycopg2 stages on vulcan).
• Performance (final sweep 2026-06-02, median of 3). Sharded apply path (SP-Perf-A-SHARD-APPLY) delivers 14.71M ops/sec at K=8 (3.00× the 4.91M K=1 baseline, sub-µs p50; K=16 → 16.24M); scan-side companions (SP-Perf-A-SHARD-SCAN / -FASTPATH / -POOL-SCALEOUT / -LOCAL-INDEX-FUSION) close the scan + find-by side. The OLTP-bracket losses (RO, RW) are CLOSED — KesselDB beats Postgres on 6 of 8 cross-DB workloads (YCSB-C 63.75×, YCSB-B 7.26×, YCSB-A 1.16×, oltp-RO 6.02×, oltp-WO 4.91×, oltp-RW 2.30× — only TPC-H Q1 2.16× + Q6 3.09× remain losses, both with named follow-up SP-JIT-Aggregate). TPC-H Q6 design floor (≥400 q/s) AND stretch (≥500 q/s) both still MET (544.59 q/s) via the 5-arc Analytic-Plan → Analytic-Plan-MULTI → Hash-Agg → Hash-Agg-Tune → WHERE-VM-Specialise chain. The final sweep re-measured every headline row on the final binary for internal consistency; oltp-WO/RW landed slightly below their prior single-arc peaks (5.2×→4.91×, 2.66×→2.30×) under live sibling-agent load — reported honestly. SQLite not re-run (vulcan root fs was 100% full; KesselDB MemVfs + Postgres docker unaffected). Raw: docs/benchmarks/finalbench-2026-06-02-*.
• Nullable columns render as SQL NULL over the PG wire (SP-PG-NULL-INT-RENDER, 2026-06-03). A nullable column omitted at INSERT, or set to an explicit NULL, now reads back as a real PG NULL (psycopg2 None) for BOTH SELECT * AND projection-list SELECT col — previously a projection rendered an omitted nullable int as 0 (text as empty), a silent data-correctness bug. Root cause was the engine's narrow Op::SelectFields projection stream carrying no null mask; the fix re-issues a non-sorted projection as SELECT * (full records, which carry the on-disk null bitmap) and re-projects in the gateway — a PURE render-layer change, no storage/wire/Op format change, so the determinism oracles stay byte-identical. Generic across kinds (int + text + numeric); NOT-NULL / PK / BIGSERIAL columns keep their real values. Explicit NULL literal support added to INSERT … VALUES. New psql smoke scripts/sppgnullintrender-smoke.py (7/7 psycopg2 stages on vulcan); the relationships (4/4), realapp (8/8), and fk-enforce (7/7) smokes stay green.
• DDL FOREIGN KEY now ENFORCED (SP-PG-DDL-FK-ENFORCE, 2026-06-03). A FOREIGN KEY (col) REFERENCES tbl [(col)] [ON DELETE …] in CREATE TABLE (table-level or inline col … REFERENCES tbl(col)) ENFORCES referential integrity: a non-NULL child FK with no matching parent → SQLSTATE 23503; NULL allowed; ON DELETE NO ACTION/RESTRICT/CASCADE/SET NULL/SET DEFAULT honored. Wiring arc — the engine FK machinery (SP6 + SP11) pre-existed; the DDL parser now captures the FK BY NAME, threads it through CreateType in a marker-guarded ADDITIVE trailer (no-FK CREATE TABLE byte-identical → determinism preserved), and the engine resolves names→ids + registers it at apply through the same path Op::AddForeignKey uses. Forward reference / unknown column → clean DDL error, no half-created type. The ORM relationships + realapp smokes pass UNDER enforcement (dependency-ordered seeds satisfy it). Deferred: composite FKs, ON UPDATE actions.
• Multi-column GROUP BY — composite group keys (SP-PG-SQL-GROUP-MULTI-COL, 2026-06-04). SELECT region, category, COUNT(*), SUM(amount) FROM sales GROUP BY region, category groups by the TUPLE of N columns, the cross-tab analytics query. Plain single-table AND binary-join; composes with HAVING / ORDER BY (aggregate or first group col) / LIMIT / OFFSET. Marker-guarded additive extra_group_fields on Op::GroupAggregate / Op::GroupAggregate Multi / JoinGroupAgg; SM builds a COMPOSITE key (primary ++ each extra's fixed-width bytes — deterministic total order) and emits each extra value as [u32 len][value] after the primary key, before the aggregates. A SINGLE- column GROUP BY is BYTE-IDENTICAL (Op frame + result stream) ⇒ determinism oracles untouched. Scatter merge threads the extra-col count so K>=2 merges composite groups. 3+ table multi-join GROUP BY is the named follow-up. Live vulcan psql smoke: 7/7 stages PASS; the SP-PG-SQL-PLAIN-GROUP-RENDER + SP-PG-SQL-GROUP-SORT-LIMIT single-column regression smokes stay green.
• RIGHT + FULL outer joins — full join-type matrix (SP-PG-SQL-RIGHT-FULL-JOIN, 2026-06-03). RIGHT [OUTER] JOIN and FULL [OUTER] JOIN complete the INNER / LEFT / RIGHT / FULL matrix on a binary join. RIGHT = matched pairs + unmatched-right rows (a.* NULL); FULL = LEFT results + unmatched-right rows. Combined column order stays a.* ++ b.* for every flavour (the JOIN drive direction is swapped, NOT the output order); NULL-filled columns read back as SQL NULL (Python None). JoinType gained Right (wire tag 2) / Full (tag 3) — purely additive (Inner byte-identical, Left = tag 1 unchanged), no new struct field, determinism oracles green. Row order is deterministic (matched/unmatched-left in scan order, then unmatched-right in right-table scan order). RIGHT/FULL compose with WHERE/ORDER BY/LIMIT/OFFSET/GROUP BY/ aliases like LEFT; pg-gateway render_join_result needed NO change (same KTR1 stream shape). RIGHT/FULL on a 3+ table CHAIN is the named follow-up (rejected cleanly; INNER chains keep working). Live vulcan psql smoke: 9/9 stages PASS.
• Table aliases in JOIN queries (SP-PG-SQL-JOIN-ALIAS, 2026-06-03). SELECT u.name, p.title FROM users u JOIN posts p ON u.id = p.user_id (and the AS form) now resolve — the SQLAlchemy/Django/Rails form. An alias→table map built from the FROM/JOIN clause resolves every qualifier (projection, ON, WHERE, ORDER BY, GROUP BY) to the full table name, for binary AND multi-table (3+) joins. Resolution is entirely in kessel-sql, so an aliased join compiles to the IDENTICAL wire Op as its full-table-name twin (no determinism risk, pg-gateway unchanged) and full-name qualifiers keep working (back-compat). Duplicate/ambiguous alias, alias shadowing a table, and unknown qualifier are clean errors; a self-join under two aliases of the SAME table is the named follow-up SP-PG-SQL-SELF-JOIN. Live vulcan psql smoke: 8/8 stages PASS.
• Chained N-way joins (SP-PG-SQL-MULTI-JOIN, 2026-06-03). 3+ table chained INNER equi-joins (users JOIN posts JOIN comments) work end-to-end over the PG wire — Op::Join gained an additive, marker-guarded extra_joins: Vec<JoinStep>; the engine folds each step into the combined KTR1 row set; WHERE/ORDER BY/LIMIT/OFFSET/SELECT * apply over the full combined schema. Empty extra-joins ⇒ byte-identical to a binary join. INNER chains only (LEFT-in-chain + GROUP-BY-over-chain are named follow-ups). Table aliases now resolve via SP-PG-SQL-JOIN-ALIAS (above).
• PostgreSQL ORM compatibility. SP-PG-EXTQ V1 (Extended Query) + V2 hardening (SP-PG-EXTQ-BIN + SP-PG-EXTQ-BIN-RESULTS + SP-PG-EXTQ-CAST + SP-PG-EXTQ-DESCRIBE-VERSION + SP-PG-SQL-PAREN-VALUES + SP-CHAR-PAD-COMPARE) closed every PARTIAL row on the ORM compat matrix. psycopg2 ✓ SQLAlchemy 2.0 ✓ psycopg3 ✓ asyncpg ✓ pgJDBC ✓ (real-driver verified on vulcan in both simple AND extended modes by SP-PG-JDBC-SMOKE). SP-PG-SQL-ORM-PARSE (2026-06-02) extends this to the declarative-ORM layer: a real SQLAlchemy 2.0 declarative-model CRUD workload (create_all DDL → multi-row INSERT → qualified-column SELECT/filter → by-PK UPDATE/DELETE) now passes 7/7 end-to-end (was 2/8) — qualified columns (t.col), explicit projection-list render, and = ANY (ARRAY[…]) all lit. SP-PG-SERIAL-RETURNING (2026-06-02) closes the last big gap: deterministic autoincrement (BIGSERIAL/SERIAL PK) + INSERT … RETURNING id. An ORM model declared WITHOUT an explicit id (the real-world default — autoincrement=True) now does full CRUD and reads the DB-assigned id back: SQLAlchemy autoincrement smoke 6/6 on vulcan. The sequence counter lives IN THE DIGEST, advanced only on the apply thread ⇒ replicated + crash-safe (3-replica byte-identity proven). SP-PG-RETURNING-MULTIROW-STAR (2026-06-03) closes the zero-config gap: KesselDB now works with SQLAlchemy's DEFAULT engine config (no use_insertmanyvalues=False). The DEFAULT batches a flush into ONE multi-row INSERT RETURNING; the gateway desugars SQLAlchemy's insertmanyvalues form to plain multi-row VALUES, surfaces N assigned ids (OpResult::CreatedMany), and RETURNING * expands to all columns. DEFAULT-config CRUD 5/5 on vulcan — "pip install, point at KesselDB, it just works". SP-PG-ORM-RELATIONSHIPS (2026-06-03) lights up the relational core: a real SQLAlchemy 2.0 two-model FK relationship (Author 1—N Book, relationship() + ForeignKey) — FK DDL, cascade insert, JOIN query, lazy-load — works 4/4 on vulcan. The gateway now renders the engine's inner-equi-Op::Join result (qualified projection + SELECT *); FK constraints in CREATE TABLE parse (accept-and-skip).
• PG COPY. SP-PG-COPY V1 (text) + SP-PG-COPY-CSV V1 + SP-PG-COPY-BIN V1 deliver the wire shape every pg_dump/pgloader/pg_bulkload/ Airbyte/Fivetran/Stitch binary-bulk-loader hard-requires. SP-PG-COPY-BULKAPPLY V1 lifts ingest 181.9× (~285 → 51,840 rows/sec).
• Cloud deploy. SP-DX-superior (Dockerfile + ghcr.io/hassard0/kesseldb
  • embedded Rust example + CLI error-class hints) + SP-Cloud-Deploy (Helm chart + fly.toml) shipped, kind-verified end-to-end on vulcan.
• Correctness. SP-CLUSTER-FLAKE T2 root-cause fix: Node::submit* retries transient ViewChange → Unavailable the same way production ClusterClient does. The long-standing CI flake is GONE.

Latest arc deliveries on top of that baseline (most-recent first): SP-PG-ORM-RELATIONSHIPS (2026-06-03, DONE) — validates a real SQLAlchemy 2.0 multi-table FK-relationship workload (Author 1—N Book) end-to-end on vulcan: 4/4 (FK DDL / cascade insert / JOIN query / lazy-load). Two surgical fixes: kessel-sql accept-and-skips FOREIGN KEY(col) REFERENCES tbl(col) (+ inline REFERENCES, ON DELETE/UPDATE) so create_all of a child table parses; the PG-wire gateway renders the engine's self-describing inner-equi-Op::Join (KTR1) result — decoding the embedded combined schema + mapping the qualified projection (SELECT authors.name, books.title … AND SELECT *). The relational core (FKs + joins) now composes through a real ORM. Determinism preserved (VSR seed-7 oracle PASS; FK DDL compiles byte-identical, JOIN render is pure). Named follow-ups: SP-PG-DDL-FK-ENFORCE, SP-PG-SQL-OUTER-JOIN, SP-PG-SQL-MULTI-JOIN. SP-PG-SQL-JOIN-WHERE (2026-06-03, DONE) — filtered inner joins (SELECT a.name, b.title FROM a JOIN b ON a.id = b.aid WHERE b.title = $1), the most common real-app join beyond bare joins (SQLAlchemy query.join(Book).filter(Book.title == x)). Op::Join gained an optional kessel-expr filter program over the COMBINED (a++b) schema; the engine joins then filters each combined row in-place. kessel-sql compiles the qualified WHERE after the ON clause against the combined field layout (a.x → left, b.y → right; bare col by suffix with ambiguity error); AND/OR/NOT/ IN/BETWEEN/LIKE + params all ride for free. Gateway render reused (fewer combined rows). Additive wire change (trailing optional filter — bare join byte-identical to the pre-arc frame). Filtered SQLAlchemy join smoke 7/7 on vulcan; determinism preserved (VSR seed-7 + 3-replica oracles PASS — the filter is a pure function of the combined row). Named follow-up: SP-PG-SQL-JOIN-ORDERBY (JOIN … WHERE … ORDER BY/LIMIT). SP-PG-SQL-OUTER-JOIN (2026-06-03, +5 KATs, DONE) — LEFT [OUTER] JOIN (SELECT a.name, b.title FROM a LEFT JOIN b ON a.id = b.aid), the join every real ORM emits for an OPTIONAL relationship (SQLAlchemy isouter=True). Op::Join gained a join_type (Inner | Left); LEFT mode emits EVERY left row, and a left row with no right match comes back ONCE with all b.* fields NULL. The combined KTR1 null bitmap carries the NULLs, so the gateway renders the PG i32 -1 sentinel with ZERO render change (decode_record + encode_data_row already handle NULL). kessel-sql parses LEFT [OUTER] JOIN; the three join-shape detectors learn the prefix. LEFT + WHERE on a b.* col drops the unmatched rows (PG semantics). Additive wire change (join-type tag appended only when non-Inner — every INNER join byte-identical to the pre-arc frame; unknown tag rejected at decode). vulcan smoke: LEFT JOIN over {tolkien, orphan} × {lotr→tolkien} returns 2 rows incl. (orphan, NULL). Determinism preserved (VSR seed-7 + 3-replica oracle PASS — unmatched rows emit in left-key scan order). Named follow-ups: SP-PG-SQL-RIGHT-JOIN, SP-PG-SQL-FULL-JOIN (DONE — see below), SP-PG-SQL-MULTI-JOIN. SP-PG-SQL-RIGHT-FULL-JOIN (2026-06-03, DONE) — RIGHT [OUTER] JOIN + FULL [OUTER] JOIN complete the INNER/LEFT/RIGHT/FULL matrix on a binary join. JoinType gained Right (wire tag 2) / Full (tag 3) — purely additive (Inner byte-identical, Left = tag 1 unchanged), no new struct field. RIGHT = the LEFT logic with the drive SWAPPED: every right row appears, an unmatched right row emits with a.* NULL — but the OUTPUT column order stays a.* ++ b.* (drive direction swapped, NOT column order). FULL = LEFT results + the unmatched-right rows (no duplicate of the matched pairs). Deterministic row order: matched/unmatched-left in left-key scan order, then unmatched-right in right-table scan order (locked by KATs). kessel-sql parses RIGHT/FULL [OUTER] JOIN (+ INNER JOIN) in the base join and every join-shape detector; aliases keep working. pg-gateway render_join_result UNCHANGED (same KTR1 stream shape; NULL a.*/b.* render as PG i32 -1 → Python None). RIGHT/FULL compose with WHERE/ORDER BY/LIMIT/OFFSET/GROUP BY like LEFT. RIGHT/FULL on a 3+ table CHAIN is rejected (named follow-up; INNER chains keep working). vulcan psql smoke 9/9: INNER (matched only), LEFT (+orphan author NULL), RIGHT (+homeless book, a.name None, order a.,b.), FULL (both + no dup). Determinism oracles PASS. Named follow-up: SP-PG-SQL-OUTER-CHAIN (RIGHT/FULL in a 3+ table chain). SP-PG-SQL-JOIN-QUERY (2026-06-03, +11 KATs, DONE) — ORDER BY / LIMIT / OFFSET over join results (SELECT a.name, b.title FROM a JOIN b ON a.id=b.aid [WHERE …] ORDER BY b.created LIMIT 20 OFFSET 40), the ubiquitous paginated-list-view shape. COMPOSES the SP23 (Op::SelectSorted) sort/page machinery with the combined join rows: Op::Join gained additive order_by / limit_n / offset_n fields; the engine STABLE-sorts the surviving combined rows by a qualified column (from either table) via a NULL-aware, kind-aware comparator (CHAR-pad-trimmed, mirroring SP23's cmp_field), then paginates. Both apply arms share ONE apply_join helper. kessel-sql resolves the qualified ORDER BY column against the combined (a++b) schema; a bare JOIN … LIMIT n keeps the legacy pre-sort limit (wire-identical), ORDER BY/OFFSET route to the post-sort fields. LEFT-join NULL sort values order NULLS LAST for ASC / NULLS FIRST for DESC (PG default). Additive page block, marker-guarded, absent for every non-paginated join ⇒ byte-identical; bad marker rejected at decode. vulcan smoke: JOIN … ORDER BY b.title LIMIT 2 → hobbit, lotr (sorted + paginated). Determinism preserved (stable sort + deterministic scan-position tiebreak; seed-7 + 3-replica oracle PASS). Named follow-ups: SP-PG-SQL-JOIN-ORDERBY-MULTI, SP-PG-SQL-JOIN-ORDERBY-EXPR, SP-PG-SQL-JOIN-AGG, SP-PG-SQL-JOIN-NULLS-ORDER. SP-PG-SQL-JOIN-AGG (2026-06-03, +13 KATs, DONE) — GROUP BY + aggregate over a join (SELECT a.name, COUNT(b.id) FROM a JOIN b ON a.id=b.aid GROUP BY a.name), the dashboard "count related rows per parent" query. COMPOSES the SP22 / SP- Analytic-Plan-MULTI group-aggregate fold with the combined join rows: Op::Join gained ONE additive field group_aggregate: Option<JoinGroupAgg> (combined-schema group_field + Vec<(kind, field_id)>). The engine groups the surviving combined Vec<Value> rows into a BTreeMap (ascending key order ⇒ deterministic) + folds the aggregates per group over the DECODED Values, emitting the [u32 ngroups]… group- aggregate result (the GroupAggregateMulti shape). NULL semantics fall out of the Value fold: COUNT(b.id) on a LEFT-join unmatched parent counts 0 (NULL b.id not counted) but COUNT(*) counts 1 (the row exists) — exact PG LEFT-JOIN-COUNT. COUNT(*) uses a COUNT_STAR_FIELD sentinel; qualified COUNT(b.id) disambiguates id across tables. Both apply arms share the fold (RO-Txn == apply). The PG gateway gains the FIRST group-aggregate render (render_join_group_aggregate + join_group_aggregate text helper): RowDescription [group col OID, agg int8] + one DataRow per group. Additive marker-guarded ga block ⇒ every non-grouped join byte- identical; bad marker rejected at decode. vulcan smoke: SELECT author.name, COUNT(book.id) … GROUP BY author.name → tolkien 2, lewis 1. Determinism preserved (BTreeMap ascending key + associative fold over deterministic scan order; seed-7 + 3-replica oracle PASS). Named follow-ups: SP-PG-SQL-HAVING, SP-PG-SQL-JOIN-GROUP-MULTI, SP-PG-SQL-JOIN-AGG-3TABLE, SP-PG-SQL-JOIN-AGG-ORDERBY-AGG. SP-PG-SQL-HAVING (2026-06-03, +3 KATs, DONE) — HAVING <AGG>(...) <cmp> <literal> filters aggregate GROUPS after grouping (SELECT a.name, COUNT(b.id) FROM a JOIN b ON … GROUP BY a.name HAVING COUNT(b.id) > 2, and the plain SELECT col, COUNT(*) FROM t GROUP BY col HAVING COUNT(*) >= 3). Spans all three group-aggregate paths: Op::GroupAggregate, Op::GroupAggregateMulti, and Op::Join's JoinGroupAgg. New HavingPred { agg_index, op, value: i128 } (keep(results) == results[agg_index] <op> value) added as ONE additive, marker-guarded Option<HavingPred> field on each. Byte-identity preserved: the HAVING block is emitted ONLY when present (tag-22 forces the range-preds length prefix only when HAVING is set), so every no-HAVING frame is BYTE-IDENTICAL to pre-arc; a non-1 HAVING marker is rejected at decode. The SQL layer parses HAVING after GROUP BY, matches its aggregate to a PROJECTED aggregate by (kind, arg field) → agg_index, and rejects a HAVING aggregate not in the SELECT list (V1). Lexer gained the SQL-standard <> inequality (both <> and != map to one opcode). The engine applies HAVING on the single deterministic apply thread over the already- deterministic per-group result, BEFORE order/limit paging (a pure function of the input rows). Gateway needs NO change — render_join_group_aggregate decodes [u32 ngroups]… so fewer surviving groups render fewer rows. vulcan psql smoke (HAVING over JOIN): baseline 3 groups → HAVING COUNT(book.id) > 2 → 1 group {tolkien:3}; >= 2 → 2 groups; = 1 → {lonely:1}; <> 3 → 2 groups; > 99 → 0 groups. Determinism preserved (seed-corpus + 3-replica byte-identity oracle PASS). V1 scope: the HAVING aggregate MUST be in the projection; HAVING over an aggregate not selected, over the group key, or on a scalar (no GROUP BY) are named follow-ups (SP-PG-SQL-HAVING-EXTRA-AGG, SP-PG-SQL-HAVING-KEY). SP-PG-SQL-PLAIN-GROUP-RENDER (2026-06-03, +3 KATs, DONE) — render a PLAIN (non-JOIN) GROUP BY group-aggregate SELECT over the PG wire (SELECT category, COUNT(*) [AS n] [, SUM/AVG/MIN/MAX(col)] FROM products GROUP BY category [HAVING …]). The planner + SM already compiled/executed plain GROUP BY (Op::GroupAggregate / Op::GroupAggregateMulti) and HAVING already filtered at the SM layer, but the gateway's render_select_got only routed group-aggregates through render_join_group_aggregate (which REQUIRES a JOIN), so a plain group-aggregate fell through to the bottom render error (0A000 only renders SELECT *). New kessel_sql::plain_group_aggregate(sql) -> Option<PlainGroupAggProj> recognizer (returns Some ONLY for a plain group-aggregate — None for JOIN-agg, single scalar agg, plain projection, and no-GROUP-BY shapes, so every existing render path is byte-untouched) + render_plain_group_aggregate (decodes the value-only group stream [u32 ngroups][u32 keylen][key][16B i128 × n_aggs]…, types the group key from the FROM-table schema, types aggregate OIDs: COUNT/SUM → int8, AVG → numeric, MIN/MAX → source-column type). Render-only — NO Op or wire-format change, so corpus / partition / 3-replica byte-identity is untouched. V1 caveat (NOW RESOLVED by SP-PG-SQL-GROUP-SORT-LIMIT, below): a trailing ORDER BY … LIMIT … OFFSET … on a plain GROUP BY was parsed but not yet engine-applied — it is now sorted + windowed by the engine. vulcan psql smoke (scripts/sppgsqlplaingrouprender-smoke.py): the headline SELECT category, COUNT(*) FROM products GROUP BY category ERRORED on pre-fix origin/main and renders {books:3, gadgets:1, toys:2} post-fix; multi-agg (COUNT/SUM/AVG/MIN/MAX) + HAVING also PASS. SP-PG-SQL-GROUP-SORT-LIMIT (2026-06-03, +3 KATs, DONE) — ORDER BY / LIMIT / OFFSET on a PLAIN (non-JOIN) GROUP BY now take effect in the engine (closes the caveat above). Op::GroupAggregate / Op::GroupAggregateMulti gained an additive, marker-guarded sort: Option<GroupSort> (GroupSortTarget::{Key, Agg(i)} + desc + limit/offset), mirroring the HAVING marker-guard and the JOIN order_by/limit_n/offset_n. The ORDER BY target resolves to a projected aggregate (alias ORDER BY n, position ORDER BY 2, or expression ORDER BY COUNT(*)) or the group key (ORDER BY g / ORDER BY 1); a shared emit_group_results helper sorts by the i128 aggregate value (or raw key bytes), reverses for DESC with an ascending-key tie-break, then applies OFFSET-then-LIMIT, AFTER HAVING (filter → sort → offset → limit) on the single deterministic apply thread. Byte-identity: the sort block is emitted ONLY when present (tag-22 forces the range-preds length prefix + a no-HAVING anchor only when HAVING/sort is set), so a no-ORDER BY/LIMIT/OFFSET frame is BYTE-IDENTICAL to pre-arc; a non-1 sort marker or bad target tag is rejected at decode. Every Op::GroupAggregate{,Multi} construction site (proto/sm/sql/read_pool/sharded_engine/parallel_reads_oracle/bench) updated with sort: None; corpus / partition / 3-replica byte-identity oracles green. Gateway needs NO change — render_plain_group_aggregate emits DataRows in engine order. vulcan psql smoke (scripts/sppgsqlgroupsortlimit-smoke.py): ORDER BY COUNT(*) DESC → books(4), gadgets(3), toys(2), misc(1) (descending count, NOT key order — pre-fix returned all 4 in key order); LIMIT 2 → top 2 only; LIMIT 2 OFFSET 1 → the right window; ORDER BY category ASC (key sort) + HAVING + ORDER BY SUM(price) DESC + LIMIT also PASS. V1 scope: single group column + single ORDER BY target; ORDER BY over a JOIN group-aggregate is the named follow-up SP-PG-SQL-JOIN-AGG-ORDERBY-AGG. SP-PG-ORM-REALAPP (2026-06-03, CAPSTONE, +3 KATs, DONE) — the headline real-world-readiness test: a realistic THREE-model SQLAlchemy 2.0 BLOG app (User 1—N Post 1—N Comment, FKs + relationship(), insertmanyvalues batching ON) exercising the full query range a real app uses, back-to-back. 8/8 stages PASS on vulcan, every query returning REAL data: schema (3 tables, 2 FKs) / multi-level cascade seed / Q1 JOIN / Q2 filtered JOIN / Q3 GROUP-BY-COUNT over JOIN / Q4 ORDER-BY+LIMIT / Q5 lazy relationship nav / Q6 UPDATE+DELETE. The first run surfaced two precise gaps, each closed by a SURGICAL fix (no engine apply / Op wire change): (1) kessel-sql lexer now handles the SQL-standard doubled-quote string escape 'bob''s post' → the previous lexer truncated at the first inner ', breaking ANY app with an apostrophe in its data (this unblocked the seed + the JOIN reads); (2) the gateway renders a projection-list SELECT with ORDER BY (which lowers to Op::SelectSorted, returning FULL records with the projection dropped at the engine layer) by decoding the full records + re-projecting requested columns with proper null-bitmap NULL fidelity. Determinism preserved (kessel-sql 135

• gateway 1003 + select_sorted_is_deterministic + VSR seed-7/3-replica oracles all PASS). No NEW follow-ups required — the blog app is 8/8. Transcript: docs/superpowers/sppgormrealapp-smoke-2026-06-03.txt. SP-PG-DJANGO-COMPLETE (2026-06-03, +14 KATs, DONE) — closes the TWO named gaps the quoted-ident arc left, taking the Django 6 ORM to full CRUD 8/8 on vulcan (was 6/8). SP-PG-DDL-IDENTITY: the CREATE TABLE column-modifier run is now order-independent and accepts <col> bigint GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( seq opts ) ] — Django 6's default BigAutoField PK DDL — as a pure parser-front alias onto the proven SP-PG-SERIAL-RETURNING deterministic autoincrement counter (sequence options parsed-and-ignored in V1; no SM/ catalog/proto change, so determinism is byte-identical to BIGSERIAL). SP-PG-SQL-AGG-ALIAS-RENDER: parse_agg captures an optional AS alias; the new select_aggregate text-helper detects a single scalar aggregate over a FROM table, and the gateway's render_select_got Shape 0 decodes the engine's 16-byte LE i128 Op::Aggregate result as RowDescription(alias or lowercase function name) + ONE DataRow + CommandComplete("SELECT 1") — what Django's .count()/.aggregate() emit (SELECT COUNT(*) AS "__count" FROM "t"). HEADLINE: Django ORM full CRUD 8/8 — connect, schema_create (IDENTITY), INSERT autoincrement (pk=1), SELECT all, get-by-PK, UPDATE, DELETE + trailing .count() (remaining count=0) all PASS. SQLAlchemy stays 7/7 (no regression). That is TWO production Python ORMs fully working against KesselDB. Determinism preserved (IDENTITY reuses the digest-covered apply-thread SERIAL counter; aggregate render is read-only). Transcript: docs/superpowers/sppgdjangocomplete-django-smoke-2026-06-03.txt. SP-PG-SQL-QUOTED-IDENT (2026-06-03, +20 KATs, DONE_WITH_CONCERNS) — the P0 keystone that unblocks the Django ORM. Django UNCONDITIONALLY double-quotes EVERY SQL identifier ("smokeapp_author"."id", "name") and kessel-sql's lexer rejected " with unexpected char '"', so the Django ORM was stuck at 2/8 even though the engine/data path was proven Django-ready. The lexer now accepts "ident" as a SQL-standard delimited identifier (case-preserving, "" escape, zero-length + unterminated rejected) everywhere a bare identifier works — table, column, qualifier, in DDL/DML/projection/WHERE/SET/RETURNING. Quoted idents lower to the SAME Tok::Ident as the bare spelling, so quoting is transparent at the compiled-Op layer and Django's quoted DDL/DML round-trip on the same catalog names (determinism preserved: quoted == bare ⇒ same Op). The gateway-side raw-SQL scanners that don't already skip quoted idents (cast stripper + literal-cast validator + insertmanyvalues find_kw) were taught to skip "…" regions so a ' or :: INSIDE a quoted identifier can't mis-pair the scanner. HEADLINE: Django ORM advanced 2/8 → 6/8 on vulcan (+INSERT autoincrement+RETURNING, SELECT, get-by-PK, UPDATE — every genuine ORM CRUD op now executes; the unexpected char '"' boundary is gone). SQLAlchemy stays 7/7 (no regression). The two residual Django gaps are pre-named follow-ups, NOT quoting: SP-PG-DDL-IDENTITY (default PK GENERATED … AS IDENTITY DDL spelling) and SP-PG-SQL-AGG-ALIAS-RENDER (SELECT COUNT(*) AS "__count" — the quoted DELETE itself passes; only the trailing .count() trips). Transcript: docs/superpowers/sppgsqlquotedident-django-smoke-2026-06-03.txt. SP-PG-SQL-DML-GENERAL (2026-06-03, +23 KATs, DONE) — completes the CRUD-with-predicates story. UPDATE/DELETE previously worked ONLY by primary key (WHERE id = n); real apps + ORMs need arbitrary WHERE predicates and multi-row mutation (UPDATE users SET active = false WHERE last_login < $1, DELETE FROM t WHERE status = 'expired') plus UPDATE … RETURNING * (optimistic concurrency). Path A (no engine/ proto surgery): the server resolves the matched ids on the leader via Op::QueryExpr (the same predicate VM SELECT uses, sorted output ⇒ deterministic), then replicates ONE concrete Op::Txn of per-id Op::UpdateSet/Op::Delete — same determinism guarantee as the by-id RMW, with full per-row index/constraint/trigger maintenance and atomic all-or-nothing rollback (a UNIQUE violation on any matched row applies ZERO rows). The gateway surfaces the real UPDATE N/DELETE N count and renders RETURNING <cols>|* (post-mutation rows for UPDATE, deleted rows for DELETE); by-PK WHERE id = n RETURNING * is routed through the same read-back path. Cluster mode supports the count path via a Cont::DmlWhere VSR continuation. seed-7 3-replica byte-identity green. HEADLINE: general-WHERE UPDATE + DELETE + RETURNING all work on vulcan (UPDATE 2 / DELETE 2 multi-row counts; RETURNING returns affected rows). SP-PG-ORM-DJANGO (2026-06-03, +1 KAT, DONE_WITH_CONCERNS) — validates a real Django 6.0 ORM workload (the OTHER dominant Python ORM) against KesselDB on vulcan. HEADLINE: connect now PASSES — a surgical set_config('TimeZone', …) connection-init intercept (mirrors the existing current_setting hook in pg_catalog::synthesize) clears the FROM-less-SELECT that Django's _configure_timezone issues on every connect, which previously killed the entire Django path before any ORM op ran. The ORM CRUD surface then funnels through ONE clean boundary: Django UNCONDITIONALLY double-quotes every identifier and kessel-sql's lexer rejects " (unexpected char '"'). Fed unquoted/BIGSERIAL SQL, every Django-shaped op (autoincrement INSERT+RETURNING, qualified SELECT, by-PK UPDATE/DELETE) PASSES — so the engine path is Django-ready and the gap is purely the SQL text shape. Smoke 2/8 stages; single P0 follow-up SP-PG-SQL-QUOTED-IDENT unblocks the rest (then SP-PG-DDL-IDENTITY, SP-PG-SQL-AGG-ALIAS-RENDER, SP-PG-DJANGO-INTROSPECT, SP-PG-SAVEPOINT). Transcript: docs/superpowers/sppgormdjango-smoke-2026-06-03.txt. SP-PG-RETURNING-MULTIROW-STAR V1 (2026-06-03, +20 KATs, DONE) — closes the zero-config SQLAlchemy milestone. SQLAlchemy 2.0's DEFAULT (use_insertmanyvalues=True) BATCHES a multi-object flush into ONE statement and expects N rows back; the SP-PG-SERIAL-RETURNING smoke had to disable it (use_insertmanyvalues=False). (1) proto — OpResult::CreatedMany { ids } (tag 16, additive) carries the per-row assigned ids. (2) SM — Op::Txn (multi-row INSERT compiles to one Txn since SP58) threads each inner Create's assigned serial id back as CreatedMany; fires ONLY when every inner op autoincrement-assigned (else byte-identical Ok); the counter advances N times on the apply thread ⇒ deterministic (3-replica byte-identity green). (3) kessel-sql — insert_returning recognizes RETURNING * (star sentinel) and accept-skips RETURNING col AS alias. (4) gateway — render_insert_returning emits N DataRows (one per assigned id) + INSERT 0 N; RETURNING * expands to all table columns via describe_table; a new insertmanyvalues rewrite desugars SQLAlchemy's INSERT … SELECT … FROM (VALUES …) AS sen(…) ORDER BY sen_counter RETURNING … to plain multi-row VALUES — applied BEFORE the literal-cast validator (which would reject the p0::VARCHAR projection cast). HEADLINE: SQLAlchemy DEFAULT-config CRUD 5/5 on vulcan (port 5544). Smoke: docs/superpowers/sppgreturningmultirowstar-t5-smoke-2026-06-02.txt.

SP-PG-SERIAL-RETURNING V1 (2026-06-02, +~30 KATs, DONE) — closes the two coupled named follow-ups SP-PG-SERIAL (deterministic autoincrement)

• SP-PG-RETURNING (return server-assigned values) TOGETHER. Real ORM models overwhelmingly use AUTOINCREMENT: the app omits id, the DB assigns it, and the ORM reads it back via INSERT … RETURNING id. (1) Determinism — a per-type sequence counter lives in a reserved, digest-covered storage keyspace (0xFFFF_FFF4), advanced ONLY on the single deterministic apply thread in op-number order (the proven SP79 sequencer pattern) ⇒ every replica computes the identical gap-free sequence; WAL-backed ⇒ crash + replay resumes it exactly. 3-replica byte-identity digest + seed-7 oracle green. (2) Catalog — a serial_pk + serial_field_id flag rides a second backward-compat trailer in the type-def blob (no-serial types encode byte-identically). (3) SM — a serial INSERT carries a SERIAL_SENTINEL id; the SM assigns the next counter value as the ObjectId AND patches it into the stored id field so SELECT id reads it back; returns OpResult::Created { id }. The counter advances only on the successful-write path (a rejected insert consumes no value; PG-matching gap semantics on abort). (4) kessel-sql — CREATE TABLE … id BIGSERIAL PRIMARY KEY flags the serial PK; an INSERT omitting the id autoincrements; RETURNING parsed; col AS alias projection accept-skipped (unblocks SQLAlchemy's refresh SELECT). (5) gateway — INSERT … RETURNING … emits RowDescription + DataRow(assigned values) + CommandComplete on BOTH simple- and extended-query paths. HEADLINE: SQLAlchemy autoincrement model (no explicit id) — w.id reads back 1 and 2 after commit; full CRUD 6/6 on vulcan (port 5543). Follow-up multi-row RETURNING + RETURNING * now CLOSED by SP-PG-RETURNING-MULTIROW-STAR (above). V1 out-of-scope (named): UPDATE/DELETE RETURNING, CREATE SEQUENCE DDL, non-PK SERIAL. Smoke transcript: docs/superpowers/sppgserialreturning-t5-smoke-2026-06-02.txt. SP-PG-EXTQ-PARSED-FUNCTIONS V1 (2026-06-02, +5 KATs, regression-lock only) — DIAGNOSIS arc. Investigated the named follow-up "scalar-function SELECTs (SELECT version() / current_database() / current_schema() / SELECT 1) still fall back to the text-substitute path under the typed-default regime." VERDICT: Reality A — the follow-up is REDUNDANT. Scalar functions are intercepted by pg_catalog::catalog_query_hook at the TOP of BOTH dispatch entry points (dispatch_query_with_params AND dispatch_query) BEFORE the typed/text branch and BEFORE any engine.apply_sql* / select_star_table call. For 0-param SQL preprocess_typed_params returns Some(vec![]), so the typed path is taken — and that path hooks the catalog FIRST, serving the synthesized RowDescription + DataRow + CommandComplete directly. No text concatenation, no engine round-trip, no correctness or security gap. The DESCRIBE-VERSION + CAT arcs already closed this; the named follow-up was speculative. Arc ships +5 end-to-end regression-lock KATs (Parse → Bind → Execute for version/current_database/current_schema/ SELECT 1 + re-Execute exhaustion) driven against a panic-on-engine-call test engine — a regression that routed a scalar function into apply_sql/apply_sql_with_params would PANIC. Frame counting walks the 4-byte length prefix (raw tag-byte counting was unsound — the version string "KesselDB 1.0" carries a literal D). vulcan-verified (port 5541/6541, psycopg3 3.3.4 Extended Query, both auto and explicit prepare=True): version()→'PostgreSQL 14.0 (KesselDB 1.0)', current_database()→'kesseldb', current_schema()→'public', SELECT 1→1. Full gateway suite 967 passed / 0 failed. Out-of-scope named follow-up: SP-PG-EXTQ-PARSED-FUNCTIONS-PARAM (gateway-evaluated PARAMETERIZED scalar functions upper($1)/length($1) — YAGNI; no ORM connect-probe issues them, and today they hit honest kessel-sql rejection, not a silent wrong answer). Smoke transcript: docs/superpowers/sppgextqparsedfunctions-t3-smoke-2026-06-02.txt. SP-PG-ORM-SQLALCHEMY V1 (2026-06-02, +1 KAT, DONE_WITH_CONCERNS) — the INTEGRATION validation of tonight's ~46 PG-wire arcs: a REAL SQLAlchemy 2.0 declarative-ORM CRUD workload (NOT raw cursor.execute) run end-to-end on vulcan. HONEST HEADLINE: the PG-wire SUBSTRATE composes (engine.connect + Extended Query probe PASS; VARCHAR(n) DDL, INSERT, and SELECT *[+WHERE] all PASS), but the DECLARATIVE-ORM layer does NOT yet compose — it is blocked by three SQL-SHAPE gaps the ORM emits that the kessel-sql parser / PG-wire render path don't recognise: (G1) create_all's inspector probe uses relkind = ANY (ARRAY[…]) → unexpected char '['; (G2) every ORM SELECT qualifies columns (SELECT t.id, t.name FROM t) + uses an explicit projection list, but the parser rejects qualified table.col projections AND the render path only emits SELECT *; (G3) ORM UPDATE/DELETE qualify the WHERE column (WHERE t.id = $1) → expected ID. Smoke = 2/8 ORM stages PASS. The ONE pre-named surgical fix this arc shipped: kessel-sql kind_of VARCHAR(n) → Char(n) DDL alias (mirrors the SP-PG-CAT-T8 BIGINT/INTEGER/SMALLINT/BOOLEAN aliases) — unblocks the DDL string-column path for every ORM (SQLAlchemy/Django/Rails/Diesel) + raw psql; KAT pg_varchar_alias_maps_to_char green on vulcan; verified via CREATE TABLE … name VARCHAR(32) + \d. The 3 ORM-shape blockers are larger than surgical and are NAMED as follow-ups: SP-PG-SQL-QUALIFIED-COLS (accept table.col in projection + WHERE/SET — unblocks G2-parse + G3), SP-PG-SQL-PROJECTION-RENDER (PG-wire render of an explicit projection list, not just SELECT * — unblocks G2-render), SP-PG-SQL-ANY-ARRAY (col = ANY (ARRAY[…]) — unblocks G1). Plus SP-PG-DDL-VARCHAR-UNBOUNDED (bare/CHARACTER VARYING), SP-PG-DDL-VARCHAR-NATIVE (true var-length storage), SP-PG-RETURNING / SP-PG-SERIAL (server-generated PKs, not hit by the explicit-id model but needed next), SP-PG-ORM-RELATIONSHIPS, SP-PG-ORM-ALEMBIC. NOTE: this REFINES the earlier "SQLAlchemy 2.0 ✓" ORM-compat-matrix claim — that ✓ is for the raw-driver path (conn.execute(text("SELECT * FROM t WHERE id=:id"))), which remains green; the declarative-ORM path is the boundary documented here. Closing the 3 SQL-shape arcs takes the declarative ORM from 2/8 to a full CRUD pass. Smoke transcript: docs/superpowers/sppgormsqlalchemy-t2-smoke-2026-06-02.txt. TaskList ready for completion (DONE_WITH_CONCERNS — boundary named, not all green). SP-PG-SQL-ORM-PARSE V1 (2026-06-02, +18 KATs, DONE) — closes the 3 keystone ORM-shape blockers named above + 2 surfaced DDL-spelling gaps, taking the SQLAlchemy 2.0 declarative-ORM CRUD smoke from 2/8 → 7/7 (full CRUD pass) on vulcan. (1) Qualified columns (SP-PG-SQL- QUALIFIED-COLS): kessel-sql col_ident() accepts table.col in projection / WHERE / SET / ORDER BY / GROUP BY, stripping the qualifier (lenient V1); strip_span_qualifiers keeps the index-hint span normalized so a qualified query compiles BYTE-IDENTICALLY to bare (determinism contract). (2) Projection render (SP-PG-SQL-PROJECTION-RENDER): gateway render_select_got emits an explicit projection list (SELECT c1, c2 FROM t, incl. qualified) via select_columns + emit_projected_ rows, not just SELECT *. (3) = ANY (ARRAY[…]) (SP-PG-SQL-ANY- ARRAY): lexes [/], desugars to IN→OR-of-eq (byte-identical to IN); pg_catalog hook recognizes SQLAlchemy's create_all relname-existence probe + synthesizes the existence answer. (EXTRA) ORM UPDATE/DELETE SET … WHERE [t.]id = n mapped to the id-based RMW; BIGSERIAL/SERIAL DDL aliases (→ plain int width, explicit-id model) + table-level/inline PRIMARY KEY accept-and-skip — unblocking real create_all DDL so every CRUD stage runs. All 7 ORM stages PASS end-to-end (create_all DDL, multi-row INSERT, qualified SELECT/filter, by-PK UPDATE+DELETE); 1055+ kessel-sql + gateway KATs green, zero regressions, gateway log clean. Residual follow-ups NAMED: SP-PG-SERIAL/SP-PG-RETURNING (autoincrement + RETURNING — for PK-omitting models), SP-PG-SQL-UPDATE- WHERE-GENERAL (non-PK/multi-row WHERE), SP-PG-SQL-QUALIFIER-STRICT, SP-PG-SQL-FROM-ALIAS, SP-PG-SQL-ANY-SUBQUERY, SP-PG-SQL-PROJ-EXPR, SP-PG-DDL-COMPOSITE-PK, SP-PG-ORM-RELATIONSHIPS/-ALEMBIC. Smoke transcript: docs/superpowers/sppgsqlormparse-t5-smoke-2026-06-02.txt. TaskList ready for completion (DONE). SP-PG-COPY-CSV-NUMERIC-SCI V1 (2026-06-02, +20 KATs) — text + CSV COPY into a NUMERIC-OID column (kessel-sql I128/U128/Fixed → PG OID 1700) now accepts scientific notation and expands the exponent into the canonical PG decimal text BEFORE the row reaches the engine. Grammar [+-]?(\d+(\.\d+)?|\.\d+)[eE][+-]?\d+ (mantissa with integer/integer+fractional/leading-dot-fractional + e/E case-insensitive + signed integer exponent). New copy::csv::parse_scientific_notation helper hand-rolls the decimal-point-shift expansion (no bigint dep): 1e10 → "10000000000"; 1.5e-3 → "0.0015"; 6.022e23 → "602200000000000000000000"; -3.14e2 → "-314". The new branch runs FIRST in validate_numeric_text so any e/E-bearing input routes through expansion; non-scientific inputs skip at zero cost. |exp|>100 cap surfaces as Malformed("exponent out of range") to prevent pathological digit-string allocation. Missing exponent (1e), multiple exponent markers (1ee2), malformed sign (1e+-3), non-integer exponent (1e1.5) reject as Malformed with precise reason. Trailing-dot mantissa (5.e2) is the named follow-up arc SP-PG-COPY-CSV-NUMERIC-SCI-TRAILDOT (no ORM / spreadsheet emits it in practice — rejection message carries the arc name). The pre-existing CsvNumericError::ScientificNotation variant is preserved for back-compat but is now unreachable from validate_numeric_text. vulcan-verified (port 5532/6532, fresh /tmp/kdb-target-csvnumsci build): 4-row CSV happy path (1e10 / 6e3 / -3.14e2 / 1.5e3) ingests and round-trips cleanly through the engine; validator-layer 1e1000 rejects with 22P02 malformed (exponent out of range); 1e rejects with 22P02 malformed (missing exponent). Honest engine-boundary doc: fractional-result scientific (1.5e-3 → 0.0015) passes the validator but the kessel-sql I128 storage layer only accepts integer values (same pre-existing gap V1 NaN/Infinity hits; V2 arc SP-PG-COPY-NUMERIC-BIGNUM). HEADLINE: scientific notation from ORM exports + spreadsheet auto-formatted CSV exports (pg_dump --csv, R write.csv(), np.savetxt, Excel/Sheets Save As CSV) ingests cleanly for the |exp|≤100 integer-yielding band — the V1 SP-PG-COPY-CSV-NUMERIC arc's named follow-up gap is CLOSED. Smoke transcript: docs/superpowers/sppgcopycsvnumericsci-t2-smoke-2026-06-02.txt. SP-PG-COPY-ABORT-DONE-TAIL V1 (2026-06-02, +5 KATs) — closes the pre-existing protocol-violation tail surfaced as a footnote in the SP-PG-COPY-CSV-NUMERIC T2 smoke. PG §55.2.7: when an ErrorResponse mid-CopyData aborts the COPY, the client may still flush trailing CopyData / CopyDone (c=0x63) / CopyFail (f=0x66) frames queued before observing the error. V1 dispatched those tail bytes through the top-level other => unsupported message tag arm, emitting a spurious 08P01 and CLOSING the connection per UnexpectedMessageDuringAuth. Real PG silently drains tail frames. Fix: an expecting_copy_tail: bool local in server::run_session armed when process_copy_data returns Failed; the top-level dispatch silently discards d / c / f while armed (c and f clear it; a fresh COPY-FROM start also clears it to prevent stale-flag leaks). Defensive 08P01 for stray c / f in pristine Idle preserved. vulcan-verified via psql 16 smoke (docs/superpowers/sppgcopyaborttail-t3-smoke-2026-06-02.txt): malformed-CSV COPY abort_smoke FROM STDIN fires the existing 22023 batch-flush error with zero unsupported message tag lines in the gateway log, AND a single psql session running SELECT 1 + \copy with bad CSV + SELECT * FROM abort_smoke completes all three on the SAME TCP connection (pre-fix the third statement surfaced connection to server was lost). HEADLINE: ETL loops batching multiple COPY commands no longer pay a reconnect-per-error cliff on noisy inputs. TaskList #383 ready for completion. SP-PG-EXTQ-CAST-VALIDATE-LITERAL V1 (2026-06-02, +28 KATs) — extends cast-validation from $N::TYPE placeholders to LITERAL::TYPE casts, closing the silent-strip hole the parent arcs left open: V1+COMPAT only tracked the declared OID when a $N preceded ::, so a cross-category literal cast like SELECT 'hello'::int8 was stripped to SELECT 'hello' and slipped through whenever the value never reached a typed column. New cast_stripper::find_literal_cast_mismatch(sql) -> Option<LiteralCastMismatch> does a single string/comment-aware pass and classifies the literal immediately before each :: (bare integer → INT4/INT8 by magnitude, bare float → FLOAT8, single-quoted string with '' escape → TEXT, true/false → BOOL, NULL → anytype sentinel; $N and arbitrary expressions are skipped as not-a-literal), then compares the literal's types::oid_category against the cast type's. The three dispatch entries (dispatch_query, dispatch_query_with_params, extq::dispatch_parse) call it BEFORE the strip rewrites the SQL; a cross-category mismatch surfaces ExtqError::LiteralCastMismatch { literal_oid, cast_oid, literal_category, cast_category } → SQLSTATE 42846 cannot_coerce via the same wire frame the $N validator uses, while NULL::TYPE accepts unconditionally (canonical typed-NULL idiom). strip_pg_casts + strip_pg_casts_tracked byte outputs are unchanged — the validator is purely additive, so every existing CAST / CAST-VALIDATE / COMPAT KAT passes byte-for-byte. vulcan-verified psql smoke (docs/superpowers/sppgextqcastvalidateliteral-t3-smoke-2026-06-02.txt): within-category 1::int8 + 'hello'::text accept; HEADLINE cross-category 'world'::int8 (TEXT→INT8) and true::int8 (BOOL→INT8) reject with the literal-cast 42846 message; NULL::int8 is NOT rejected by the validator (engine-level error only). Full pg-gateway lib sweep 962/962 green on vulcan at HEAD 02df4a0. TaskList #386 ready for completion. V2 follow-ups named: SP-PG-EXTQ-CAST-VALIDATE-LITERAL-EXPR (literal casts inside expressions, (1+2)::int8), SP-PG-EXTQ-CAST-VALIDATE-LITERAL-DATEPARSE ('2024-01-01'::date), SP-PG-EXTQ-CAST-VALIDATE-LITERAL-NUMSTR ('42'::int8), SP-PG-EXTQ-CAST-VALIDATE-LITERAL-MULTIWORD (multi-word type names). SP-PG-EXTQ-CAST-VALIDATE-COMPAT V1 (2026-06-02, +14 KATs) — relaxes SP-PG-EXTQ-CAST-VALIDATE's V1 strict OID equality to PG's pg_type.dat::typcategory compatibility table. V1 strict equality was correct against the V1 contract but wrong against real ORM behaviour: pgJDBC's default Long binding sends INT8 but a Java int against an ::int8 cast sends INT4 + INT8 mismatched at the wire; psycopg3 has the same shape for Python int. PG itself accepts these widenings. New helpers types::oid_category(oid) -> char (returns 'N' numeric / 'S' string / 'B' bool / 'D' date-time / 'U' unknown-or-bytea) + types::oid_castable(param_oid, cast_oid) -> bool (strict equality + omitted-OID skip + intra-category widening). extq::dispatch_bind's validator swaps strict != for !oid_castable(...); error variant + state set + first-mismatch- wins ordering byte-untouched. Cross-category mismatches (TEXT vs INT8, BOOL vs INT8, BYTEA vs TEXT) STILL reject with the same ExtqError::CastOidMismatch → 42846 cannot_coerce wire frame so the V1 silent-coercion vector stays closed; only intra-category pairs newly accept. vulcan-verified via psycopg3 PQ-layer 5-case smoke (docs/superpowers/sppgextqcastvalidatecompat-t3-smoke-2026-06-02.txt): HEADLINE INT4 param + INT8 cast accepted; symmetric INT8 + INT4 also accepted; TEXT + VARCHAR accepted; cross-category TEXT + INT8 still rejects with the exact V1 message ("cannot cast parameter $1 from type with OID 25 to declared cast type OID 20"); strict-equality INT8 + INT8 base case still works. V2 follow-ups named: SP-PG-EXTQ-CAST-VALIDATE-COMPAT-RANGE (overflow-check param value vs cast-type range, e.g. INT4 value 100000 vs INT2 cast), SP-PG-EXTQ-CAST-VALIDATE-LITERAL (also relax-and-validate literal casts), SP-PG-EXTQ-CAST-VALIDATE-CATEGORY-CROSS (accept SOME cross-category casts PG itself accepts, e.g. TEXT '42' → INT8). SP-PG-EXTQ-CAST-VALIDATE V1 (2026-06-02, +17 KATs) — closes the V1 SP-PG-EXTQ-CAST "strip + hope" silent-coercion attack vector. cast_stripper::strip_pg_casts_tracked(sql) -> (String, Vec<(usize, u32)>) extends the V1 stripper with a tracking vec pairing each stripped $N::TYPE cast with the type's PG OID; PreparedStmt.param_casts stores the pairs at Parse time; dispatch_bind rejects any mismatch between the bound parameter OID and the declared cast OID with ExtqError::CastOidMismatch which server.rs renders to SQLSTATE 42846 cannot_coerce. Skip-rule for asyncpg / psycopg3 default shape: when Parse omitted the OID hint at that position (= 0 = infer), the validator skips — the omitted hint is the client's explicit "trust the SQL" signal. vulcan-verified via psycopg3 PQ-layer 3-case smoke (docs/superpowers/sppgextqcastvalidate-t3-smoke-2026-06-02.txt): matching OID succeeds, mismatched OID rejects with exact 42846 + message naming both OIDs ('cannot cast parameter $1 from type with OID 25 to declared cast type OID 20'), omitted-OID skip-rule works. Literal-cast psql shapes (parent arc regression-guard) PASS byte-for-byte. HEADLINE: the silent-coercion vector the parent arc explicitly flagged ("V1 scope is strip + hope") is CLOSED. V2 follow-ups named: SP-PG-EXTQ-CAST-VALIDATE-COMPAT (PG type- category compatibility table instead of strict OID equality), SP-PG-EXTQ-CAST-VALIDATE-LITERAL (also validate literal casts, not just $N), SP-PG-EXTQ-CAST-VALIDATE-MULTIWORD (recognise multi-word PG type names like TIMESTAMP WITH TIME ZONE). SP-PG-COPY-CSV-NUMERIC V1 (2026-06-02, +21 KATs) — text + CSV COPY into a NUMERIC-OID column (kessel-sql I128/U128/Fixed → PG OID

1700. now validates the canonical PG decimal grammar at the gateway BEFORE the row reaches the BULKAPPLY fold. New copy::csv::validate_numeric_text accepts canonical signed decimals (with sign normalisation: +42 → 42; -0 → 0), leading-dot / trailing-dot tolerated per PG, and case-insensitive specials (nan, NaN, Infinity, INFINITY, +infinity, inf, +inf, -infinity, -inf) canonicalising to the PG mixed-case form. Malformed inputs (1.2.3, hello, --5, lone-sign, lone-dot, empty/whitespace, scientific notation) reject with a precise 22P02 invalid_text_representation naming the failing row + column + reason + V2-arc where applicable (SP-PG-COPY-CSV-NUMERIC-SCI for scientific notation). validate_numeric_fields dispatcher helper runs the validator on every NUMERIC column of every parsed row in BOTH process_copy_data_text AND process_copy_data_csv, rewriting the field bytes to the canonical form on success so the synthesized INSERT VALUES carries the normalised representation. NULL fields pass through unchanged. vulcan-verified (port 5538/6538 — port collision with sibling agent forced a shift): 6-row CSV happy path (42 / 12345 / -3 / 1000 / -50000 / +999→999) round-trips byte-equal through COPY ... TO STDOUT WITH (FORMAT csv, HEADER); validator- layer rejections surface the precise messages above; engine-side NaN/Inf I128 storage gap honestly named as a downstream V2 arc (SP-PG-COPY-NUMERIC-BIGNUM / SP-PG-NAN-IN-ENGINE). HEADLINE: text/CSV NUMERIC validation gap closes — pg_dump --csv of NUMERIC columns + analyst CSV uploads with case-insensitive specials work to the validator boundary; malformed shapes surface clean SQLSTATE-tagged errors instead of confusing generic kessel-sql parse failures. Smoke transcript: docs/superpowers/sppgcopycsvnumeric-t2-smoke-2026-06-02.txt. SP-PG-EXTQ-PARSED-BYTEA-TYPED V1 (2026-06-02, +10 KATs) — typed- path BYTEA support preserves arbitrary raw bytes (including non- UTF8 sequences like 0xFF/0xFE/0x80/isolated continuation bytes). kessel-sql gains Tok::Bytes(Vec<u8>) + Lit::Bytes(Vec<u8>) variants; rewrite_param_tokens routes Value::Blob(b) through Tok::Bytes (NO UTF-8 round-trip — the prior path's String::from_utf8_lossy(b) corrupted any byte the UTF-8 grammar doesn't accept). preprocess_binary_value(PG_TYPE_BYTEA, _) returns Some(Value::Blob(bytes.to_vec())) so BYTEA-binary uniformly flows through the typed path with INT/BOOL/TEXT/VARCHAR. vulcan-verified: psycopg3 binary-format INSERT round-trips non-UTF8 payloads (fffefd8090a0b0c0, 00...00, deadbeefcafebabe) byte-equal; psycopg2 text-format CHAR path regression-free. HEADLINE: non-UTF8 BYTEA bytes survive the typed path verbatim (was: corrupted by from_utf8_lossy to U+FFFD replacement chars). SP-PG-EXTQ-PARSED-DEFAULT V1 (2026-06-02, +11 KATs) — typed-param path becomes the gateway DEFAULT. dispatch_execute now routes through apply_sql_with_params whenever every bound parameter is typed-eligible; the text-substitution path stays as the fallback for FLOAT/TIMESTAMPTZ/NUMERIC (post BYTEA-TYPED, BYTEA binary also flows through the typed path). New PARAMETERIZED_SQL_TAG = 0xF3 admin frame carries (sql, params) to the engine thread where compile_stmt_with_params runs against the live catalog. vulcan-verified: psycopg2 + asyncpg + psycopg3 smoke regression-free; quote-injection wire test confirms the table is NOT dropped ("; DROP TABLE inj_smoke; -- stored verbatim, post- injection INSERT succeeds → 2 rows visible). HEADLINE: closes the SP-PG-EXTQ V1 §11 weak-spot #1 attack surface at the DISPATCH layer (V1 closed it at the kessel-sql + classifier layer only). SP-PG-EXTQ-PARSED V1 (2026-06-02, +31 KATs) — kessel-sql $N parameter token + compile_with_params typed-param threading + gateway classifier; closes the V1 §11 weak-spot #1 SQL-text- substitution attack surface. SP-WHERE-VM-Specialise V1 (2026-06-01, +17 KATs) — per-row WHERE evaluator compiles to a closure once per query, cutting the dominant TPC-H Q1/Q6 wall-time cost. SP-PG-SQL-PAREN-VALUES V1 (2026-06-02, +2 KAT functions / +13 assertions in kessel-sql) — closing the last residual the SP-PG-JDBC-SMOKE T2 transcript named (pgJDBC simple-mode PreparedStatement INSERT + WHERE round-trip through the real driver). SP-PG-EXTQ-DESCRIBE-VERSION V1 (2026-06-02, +18 KATs) — gateway emits RowDescription for the scalar SELECTs that pgJDBC probes at connect. SP-PG-JDBC-SMOKE V1 (2026-06-02, +0 KATs — verification-only) — real pgJDBC 42.7.4 on vulcan: CRUD chain PASS in both modes. SP-CHAR-PAD-COMPARE V1 (2026-06-02, +15 KATs) — engine-side CHAR(N) trailing-NUL/space insignificance fix surfaced by SP-PG-EXTQ-BIN-RESULTS smoke. SP-PG-EXTQ-CAST V1 (2026-06-02, +26 KATs) — ::TYPE[(args)] stripper at dispatch entry, JDBC simple-mode unblocked.

Tonight's delivery (2026-06-02) — coherent state of the union:

• Track O — SP-PG-EXTQ-PARSED (2026-06-02, V1 SHIPPED). Closes the SP-PG-EXTQ V1 §11 weak-spot #1 attack surface (SQL-text parameter substitution + '→'' escape brittleness) for every typed-path-eligible parameter. kessel-sql lexer gains Tok::Param(u16) recognizing $1..$99 as 1-based positional placeholders (T1, +7 KATs); $0 rejected (PG semantics), $100+ rejected (V1 cap), bare $ rejected (lexer is strict; the gateway-side scanner stays permissive). kessel-sql parser gains compile_with_params(sql, cat, params: &[Option<Value>]) + compile_stmt_with_params(...) entry points; the rewrite happens at the TOKEN level after lex / before parse — bound Values enter as typed tokens (Int → Tok::Int, Blob → Tok::Str, Null → Tok::Ident("NULL")) and never get concatenated into SQL text (T2, +12 KATs covering INSERT VALUES / WHERE / UPDATE SET / multi- param ordering / same-$N-twice / NULL injection / out-of-bounds rejection / no-placeholders pass-through / mixed bare-literal / Value::Uint coercion / the HEADLINE SECURITY KAT — a quote- injection payload like '; DROP TABLE t; -- in a bound parameter survives as a Value::Blob operand at the EQ comparison; the engine never sees the injected SQL because the bound bytes were carried through the AST verbatim). Internal refactor: compile()

  • compile_stmt() bodies extracted into compile_from_tokens / compile_stmt_from_tokens so params + bare paths share one parser dispatch (no double-rewrite, no shape drift). kessel-pg-gateway classifier gains preprocess_typed_params(params, formats, oids) -> Option<Vec<Option<Value>>> — returns Some(...) only when every parameter can be typed cleanly; None signals graceful fallback to the existing text-substitution path. Per-OID routing (INT2/4/8 / BOOL / TEXT/VARCHAR/BYTEA → typed; FLOAT4/8 / TIMESTAMPTZ / NUMERIC → fallback). T3 +12 KATs locking the classifier contract, including the gateway-end-to-end HEADLINE KAT (payload routes through gateway → kessel-sql → program). V1 disposition: typed path is opt-in (KAT-only exercise); default dispatch_execute still uses the text-substitution path so we don't risk a silent compat regression. Follow-up SP-PG-EXTQ-PARSED-DEFAULT flips the default after soak. Two V2+ follow-ups named: SP-PG-EXTQ-PARSED-INFER (Parse-time OID- driven type inference), SP-PG-EXTQ-PARSED-CACHE (pre-compiled AST cache to avoid re-lex/re-parse on every Execute). vulcan- verified: kessel-sql lib 64/64 (45 baseline + 7 T1 + 12 T2); kessel-pg-gateway lib 841/841 (829 baseline + 12 T3); workspace cargo build --features pg-gateway clean. HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched (engine-side improvement; the gateway routes through the same dispatch path by default). #![forbid(unsafe_code)] honored; zero new external deps. Three commits: d4d6366 (T1 design + lexer + 7 KATs), fd7fdd1 (T2 compile_with_params + 12 KATs), de9dbea (T3 gateway classifier + 12 KATs). Design: docs/superpowers/specs/2026-06-02-kesseldb-sppgextqparsed-design.md. Progress tracker: docs/superpowers/specs/2026-06-02-kesseldb-subproject-sppgextqparsed-progress.md → V1 CLOSED. TaskList #374 ready for completion.

• Track K cont. — SP-Cloud-Cluster-METRICS-EXPAND (2026-06-02, V1 ARC CLOSED — proper kesseldb_view_changes_total counter + kesseldb_replica_lag_opnum gauge + cluster-mode /v1/metrics HTTP endpoint + PrometheusRule rewrite). Closes the named V2 follow-up that the SP-Cloud-Cluster V1 T7 ship explicitly called out — the delta(kesseldb_view_number[5m]) > 5 surrogate miscounts across replica restarts because the view-number gauge resets. T1: kessel-vsr::Replica gains view_changes_total: u64 (bumped via a centralized advance_view_to helper that funnels every previous self.view = ... site — 6 in total) and last_primary_op_seen: u64 (captured from inbound Msg::Prepare, reset on view change). Public accessors view_changes_total() + replica_lag_opnum() (returns 0 on primary; saturating_sub(op_number()) on backup). 2 new vsr KATs + 27/27 existing tests stay green. T2: MetricsSnapshot grows view_changes_total + replica_lag_opnum fields (additive); metrics_writer::render emits 2 new HELP/TYPE/ sample blocks; single-node EngineHandle emits both as 0 honestly. cluster::Node::metrics_probe() returns a ClusterMetricsSnapshot via a new Ev::MetricsProbe event. cluster::serve_metrics_http(listener, node) is a minimal HTTP/1.1 server (no keep-alive, no body parsing) that serves GET /v1/metrics (Prometheus text v0.0.4) + GET /v1/health (JSON liveness) + 404 for anything else. run_cluster_cfg honors KESSELDB_HTTP_ADDR to bind the metrics endpoint as a sibling listener; SQL/Op gateway surfaces in cluster mode remain a documented V2 follow-up (the same one SP-Cloud-Cluster V1 named). 1 new cluster KAT covers the rendered surface across all three replicas. T3: PrometheusRule.yaml swaps delta(kesseldb_view_number[5m]) > 5 for rate(kesseldb_view_changes_total[5m]) > 1 — proper counter shape that survives replica restart via Prometheus's standard counter-reset detection in rate(). Adds KesselDBReplicaLag alert (kesseldb_replica_lag_opnum > 100 for 60s, severity warning); the gauge resets to 0 on every view change so planned failover does NOT page. values.yaml comment block updated to drop the V1 surrogate caveat. T4 vulcan verification: 3-replica cluster spawn (HTTP on :6330/:6331/:6332, client on :6540/:6541/:6542, peer on :6532/:6533/:6534 — the brief's 127.0.0.1:653$i client mapping collided with peer addrs on loopback so distinct ports were used). Pre-kill: all 3 replicas show view_changes_total=0, view=0; replica 0 is primary. kill the primary → sleep 4 → re-scrape: replica 1 is now primary in view 1 with view_changes_total=1 (THE HEADLINE); replica 2 still backup in view 1 with view_changes_total=1 as well. /v1/health returns the expected JSON; unknown paths return HTTP 404. Honest limits: (a) replica_lag_opnum accuracy is bounded by Prepare cadence — a quiet primary leaves the gauge stale at the last Prepare's op_number; (b) view_changes_total is per-process and resets on replica restart, which Prometheus's rate() handles via counter-reset detection; (c) the cluster-mode HTTP endpoint serves observability only (SQL/Op gateway in cluster mode is still a V2 follow-up). Invariants preserved: default single-pod path byte-identical when KESSELDB_HTTP_ADDR is unset (the default); HTTP/1.1 single-node gateway SQL/Op surfaces byte- untouched (this arc only added 2 fields, both 0 in single-node mode); WS + binary + PG-wire surfaces byte-untouched; #![forbid(unsafe_code)] honored; zero new external deps. KAT delta: +3 net (2 vsr + 1 cluster). Two commits: 92f17ae (T1+T2 — vsr counter + cluster /v1/metrics endpoint), 25ac248 (T3 — PrometheusRule swap to proper counter + new ReplicaLag alert). Progress tracker: docs/superpowers/specs/2026-06-02-kesseldb-subproject-spcloudcluster-metricsexpand-progress.md. Vulcan transcript: docs/superpowers/spcloudcluster-metricsexpand-vulcan-2026-06-02.txt. TaskList #379 ready for completion (V1 arc DONE).

• Track K cont. — SP-Cloud-Cluster T7+T8 (2026-06-02, V1 ARC CLOSED — Prometheus ServiceMonitor + PrometheusRule + USAGE + README + STATUS). Closes the SP-Cloud-Cluster V1 arc. T7 adds prometheus-operator CRDs (monitoring.coreos.com/v1 ServiceMonitor + PrometheusRule) as opt-in Helm templates gated on cluster.enabled AND monitoring.prometheus.enabled (default OFF; chart still installs cleanly in operator-less clusters). The ServiceMonitor targets the chart's existing client ClusterIP Service on the named http port (6533) at /v1/metrics. The PrometheusRule ships three alerts driven by the V1-emitted metric surface (crates/kessel-http-gateway/ src/metrics_writer.rs — kesseldb_ops_total{kind}, kesseldb_inflight, kesseldb_last_op_number, kesseldb_view_number (monotonic), kesseldb_is_primary, kesseldb_http_requests_total{path,status}, plus Prometheus-injected up{}): KesselDBClusterReplicaDown (up{}==0 for 30s — critical), KesselDBNoPrimary (sum(kesseldb_is_primary)==0 for 60s — critical), KesselDBViewChangeStorm (delta(kesseldb_view_number[5m])>5 for 5m — warning). values.yaml grew a monitoring.prometheus.* block (enabled, interval 30s, scrapeTimeout 10s, additionalLabels, rules.enabled, rules.additionalLabels). Honest metric-naming caveat: V1 does NOT emit a dedicated kesseldb_view_changes_total counter or kesseldb_replica_lag_seconds histogram; the delta(kesseldb_view_number[5m]) rule is the V1 surrogate. Named V2 follow-up arc SP-Cloud-Cluster-METRICS-EXPAND ships the proper counter + lag histogram. Verification on vulcan (helm v3.16.3): both helm lint paths clean (default mode + --set cluster.enabled=true --set monitoring.prometheus.enabled=true); object counts: DEFAULT → 1× Deployment + 1× PVC + 1× Service + 1× ServiceAccount; CLUSTER (no monitoring) → 1× StatefulSet + 2× Service + 1× ServiceAccount; CLUSTER + monitoring → adds 1× ServiceMonitor + 1× PrometheusRule; CLUSTER + monitoring with rules.enabled=false → adds 1× ServiceMonitor (no rule). T8 arc closure: USAGE.md §11.5 grew a #### Prometheus monitoring sub-section (helm upgrade invocation with operator-selector label hint, alert table, V1-emitted metric table, knobs list, V2 metric-naming caveat) + an expanded V1-limits list naming every V2 follow-up (HTTP/WS/PG gateway in cluster, Fly multi- region, online reconfig, coordinated backup). README's Deploy table grew a dedicated Kubernetes cluster row (--set cluster.enabled=true --set cluster.replicas=3 one-liner) + link to USAGE §11.5 + link to the kind primary-kill transcript. T6 (Fly multi-region) deferred out of V1 (needs a Fly account); named V2 follow-up arc retained at full priority. Invariants preserved: default single-pod render byte- identical (monitoring gated on cluster.enabled); cluster-no- monitoring render byte-identical to T5 ship; zero Rust code touched; HTTP/1.1 + WS + binary + PG-wire surfaces byte- untouched; #![forbid(unsafe_code)] honored (n/a — YAML + Markdown only); zero new external deps. KAT delta: +0 (YAML + docs only). Two commits: 501dd6a (T7 chart additions + values block), 04f0014 (USAGE + README + STATUS + progress tracker close). Progress tracker: docs/superpowers/specs/2026-06-02-kesseldb-subproject-spcloudcluster-progress.md — V1 CLOSED, T6 + METRICS-EXPAND + GEO + SHARD + BACKUP + RECONFIG + VERIFY-MULTI-NODE all named V2. TaskList #377 ready for completion (V1 arc DONE).

• Track K cont. — SP-Cloud-Cluster T1 (2026-06-02, T1 SCAFFOLD LANDED; T2-T8 MULTI-ARC CONTINUATION QUEUED). Multi-pod replicated VSR clustering — the production-deploy story on top of SP-Cloud-Deploy V1's single-pod foundation. T1 ships the design spec + Helm chart StatefulSet + headless Service + values.yaml cluster: block; T2 wires the binary CLI flags (--cluster / --replica-idx / --peer-addrs) through to kesseldb_server::cluster::spawn_node; T3-T8 are kind verify + cluster smoke (primary-kill + view-change) + Fly.io multi-region + monitoring + arc closure. Design: docs/superpowers/specs/2026-06-02-kesseldb-spcloudcluster-design.md (11 sections incl. V1 IN/OUT, Helm shape, env vars, pod entrypoint, acceptance, 10-weak-spot self-review, V2+ follow-up arcs — GEO / SHARD / BACKUP / RECONFIG / VERIFY-MULTI-NODE — all named). Helm additions: templates/statefulset.yaml (new — conditional on cluster.enabled; replicas=3 default, podManagementPolicy=Parallel, serviceName={fullname}-headless, volumeClaimTemplates supersede the single-pod PVC, entrypoint shell derives $IDX from ${HOSTNAME##*-}); templates/service-headless.yaml (new — clusterIP: None + publishNotReadyAddresses: true for VSR bootstrap before any pod is k8s-Ready); values.yaml extended with a cluster: block (enabled=false default / replicas=3 / peerAddressTemplate {name}-{idx}.{name}-headless.{namespace}.svc.cluster.local:6532 / viewChangeTimeout=5s / podManagementPolicy=Parallel); _helpers.tpl extended with a kesseldb.clusterPeerAddrs helper that expands the DNS template across 0..replicas and joins with ,; templates/deployment.yaml + templates/pvc.yaml gated so they ONLY render in single-pod mode (cluster mode uses StatefulSet + volumeClaimTemplates). Verified on vulcan (helm v3.16.3): helm lint 0 chart(s) failed in BOTH default + cluster modes; default render produces 1× Deployment + 1× PVC + 1× Service + 1× ServiceAccount (BYTE-IDENTICAL to SP-Cloud-Deploy V1 — existing installs upgrade with no diff); cluster render produces 1× StatefulSet + 2× Service (client ClusterIP + headless) + 1× ServiceAccount + 0× Deployment + 0× PVC. KESSELDB_CLUSTER_PEER_ADDRS env correctly expanded at both N=3 (3 stable DNS addrs) and N=5 (5 addrs). Headless service emits the required clusterIP: None + publishNotReadyAddresses: true knobs. Open-mode branch (auth.secretName="") still correctly drops KESSELDB_TOKEN env in cluster mode. T1 caveats (intentional, named, not vague): today's image will CrashLoopBackOff on unknown argument --cluster (clean failure mode, NOT stuck-pending — the binary CLI wire-up is T2); no live kind verify in T1 (no kind cluster running on vulcan at T1 time; deferred to T4 with the T2-extended binary; helm lint + helm template already prove the YAML scaffold is well-formed); Fly.io path is separate (Fly Machines don't have stable headless-Service-style DNS — T6 ships a Fly-specific transport using <machine-id>.vm.<app>.internal or 6PN addresses). Zero Rust code touched (YAML + Markdown only); workspace test count unchanged; default cargo build byte-identical; HTTP/1.1 + WS

  • binary + PG-wire surfaces byte-untouched; #![forbid(unsafe_code)] honored (no Rust changes); zero new external deps. Two commits this slice: c44d883 (T1 design spec + Helm scaffold + progress tracker)
  • this commit (T1 STATUS row). Progress tracker: docs/superpowers/specs/2026-06-02-kesseldb-subproject-spcloudcluster-progress.md — T1 DONE; T2-T8 multi-arc continuation QUEUED. TaskList #371 T1 done; T2-T8 queued for multi-week arc continuation.

• **Track K cont. — SP-Cloud-Cluster T2 (2026-06-02, T2 BINARY WIRE-UP

  • kind-verified).** Closes the T1 caveat (today's image CrashLoopBackOff on unknown argument --cluster) by teaching the kesseldb binary to parse the cluster-mode flags + env vars and dispatch into the existing real-TCP VSR transport that cluster.rs::spawn_node already shipped (SP38). Binary CLI: new flags --cluster, --replica-idx N, --peer-addrs A,B,C, optional --view-change-timeout T (informational in V1); CLI takes precedence over the matching KESSELDB_CLUSTER_* env vars (which the chart sets). lib.rs: new public run_cluster_cfg(client_addr, peer_listen_addr, data_dir, self_idx, peer_addrs, cfg) — binds the client + peer listeners, spawns the cluster::Node on the engine thread, and exposes the binary protocol via the auth-aware cluster::serve_clients_cfg. Refuses to start with a typed io::Error on even N or N<3 (matches the VSR fixed-size contract, before Replica::new would panic) and out-of-range replica idx. cluster.rs: new Ev::RoleProbe + Node::role_probe() returning (view, is_primary, status) so a small startup loop in the binary can emit a one-shot "elected primary" log on the role transition (the kind-verify acceptance target). serve_clients_cfg(listener, node, token) mirrors the single-node [0xFC] ++ token auth handshake so existing kessel-client / ClusterClient instances work unchanged in both open + token modes; legacy serve_clients is now a thin serve_clients_cfg(.., None) wrapper (existing tests pass verbatim). Bootstrap-race fix: resolve_peer_addrs retries every 2s for up to 120s — initial k8s StatefulSet pods occasionally start before their own headless DNS A-record is published (CoreDNS lag past publishNotReadyAddresses), and a naive to_socket_addrs errors out immediately. The retry loop logs kesseldb cluster: DNS bootstrap: ... retrying in 2s and recovers cleanly without CrashLoopBackOff. Helm chart: introduces a dedicated peer port (cluster.peerPort: 6534, also in peerAddressTemplate) so the binary doesn't bind-collide between the client port (6532) and the peer port on the same pod; statefulset.yaml exposes 6534; service-headless.yaml publishes 6534 (the headless service no longer carries the binary port — clients still use the regular ClusterIP Service which routes 6532/6533/5432). Verification on vulcan (kind v0.24.0 + helm v3.16.3): fresh kind cluster, helm install with cluster.enabled=true, all 3 pods (kesseldb-0/1/2) reach Running in ~45s; primary elects in view=0 within 1s of binary start; CRUD via primary's local port (CREATE TABLE / INSERT / SELECT) returns 42 as written; transcript at docs/superpowers/spcloudcluster-t2-kind-verify-2026-06-02.txt. Cluster tests stay green: 6/6 cluster::tests::* (three_nodes_replicate_over_real_tcp, sql_over_cluster_full_crud_and_rmw, session_retry_is_exactly_once, failover_retry_against_follower_returns_cached_reply, cluster_client_finds_primary_and_is_exactly_once, cluster_sql_cache_correct_across_ddl). Honest T2 limit (carried forward to T3-T8): the kessel CLI uses single-Client::connect, so writes routed via the round-robin ClusterIP Service can land on a backup and hit OpResult::Unavailable; the failover-aware shape is ClusterClient (already shipped + tested at SP42). T3 wires the CLI / SDK clients onto the cluster headless Service endpoint set so random-pod routing works end-to-end. Invariants preserved: default cargo build -p kesseldb-server byte-identical when --cluster is absent (main.rs dispatches through the pre-existing run_cfg path); HTTP/1.1 + WS + binary + PG-wire single-node surfaces untouched (cluster gateway surfaces are V2 follow-up); #![forbid(unsafe_code)] honored; zero new external deps. Three commits: b5db272 (CLI/env wire-up + cluster dispatch + Node::role_probe + serve_clients_cfg), f34a758 (DNS bootstrap retry loop, kind verify root-cause), eee966e (kind verification transcript). Progress tracker: docs/superpowers/specs/2026-06-02-kesseldb-subproject-spcloudcluster-progress.md — T2 DONE; T3-T8 multi-arc continuation QUEUED. TaskList #373 T2 done; T3-T8 still queued.

• Track K cont. — SP-Cloud-Cluster T3+T5 (2026-06-02, FAILOVER- AWARE CLI + kind primary-kill VERIFIED). Closes the T2 honest caveat (kessel CLI uses single-Client::connect, so writes routed via the round-robin ClusterIP Service can land on a backup and hit OpResult::Unavailable) by wiring the failover-aware ClusterClient already shipped at SP42 into the CLI's SQL path, AND end-to-end verifying it on a kind 3-pod cluster with the primary kubectl deleted mid-test. CLI (kessel): new --addrs A1,A2,... flag (comma-separated cluster addresses); when multi-addr, dispatches through ClusterClient::sql instead of single Client::sql. The --addr (singular) path stays byte- identical for single-target installs. ClusterClient: new sql(&str) method writes [0xFE] ++ utf8 (the same wire shape Client::sql writes) and retries on OpResult::Unavailable / I/O error by rotating the address index. The cluster server's apply_raw path already accepts that shape on every node and either compiles + commits (primary) or answers Unavailable (backup unable to relay) — so the client-side rotation lands the SQL on the active primary regardless of which address it dialed first. Helm chart NOTES.txt: grew a CLUSTER MODE section rendering the full kessel --addrs ... invocation with the per-pod headless DNS list + a primary-kill recovery hint (single-pod NOTES is byte-identical; gated on .Values.cluster.enabled). Two new cluster KATs: cluster_client_sql_rotates_past_followers (primary LAST in the address list; ClusterClient::sql still lands CREATE / INSERT / SELECT SUM correctly) + cluster_client_sql_commits_through_follower_port (only a FOLLOWER's client port is in the address list; the follower's server-side relay-to-primary commits DDL + 2× INSERT + SUM=300 via [0xFE] ++ sql). 8/8 cluster::tests::* green (up from 6/6 at T2). T5 live kind verify on vulcan (kind v0.24.0 + helm v3.16.3 + Docker 27.5.1, Ubuntu 24.04): fresh kind cluster, helm install cluster.enabled=true, all 3 pods Running in <60s; pre-kill INSERT(100) + SELECT SUM = 100; kubectl delete pod kesseldb-cluster-0 (the primary in view=0); within ~8s kesseldb-cluster-1 logs elected primary (view=1); next kessel --addrs ... INSERT(200) returns Ok; final SELECT SUM(v) FROM failover_smoke → = 300 (16 bytes) (100 + 200 — the headline result). Transcript: docs/superpowers/spcloudcluster-t3-t5-failover-2026-06-02.txt. Honest T3+T5 limits: cross-node exactly-once on SQL writes is NOT guaranteed (the [0xFE] ++ sql path is not session- framed because the cluster server's session-frame path is Op-only — embedded callers needing strict exactly-once should use ClusterClient::call(&Op) instead, which IS session-framed and dedupes via the replica's client_table); HTTP / WS / PG-wire gateways still not served in cluster mode V1 (V2 follow-up). Invariants preserved: kessel --addr <single> path byte- identical; HTTP/1.1 + WS + binary + PG-wire single-node surfaces untouched; #![forbid(unsafe_code)] honored; zero new external deps. KAT delta: +2 cluster KATs (8 total). Three commits: 233f4a2 (CLI --addrs + ClusterClient::sql + Helm NOTES.txt

  • 2 new cluster KATs), 7ce5250 (KAT fix — simplify failover KAT to follower-relay shape), 0d95405 (T5 kind verification transcript). USAGE §11.5 added (Kubernetes cluster mode walk- through + primary-kill failover smoke). Progress tracker: docs/superpowers/specs/2026-06-02-kesseldb-subproject-spcloudcluster-progress.md — T3 DONE, T5 DONE (T4 was folded into T2 at the prior slice); T6 (Fly multi-region) + T7 (Prometheus) + T8 (arc closure) multi-arc continuation QUEUED. TaskList #375 T3+T5 done; T6-T8 still queued.

• Track M — SP-WHERE-VM-Specialise (2026-06-01, V1 SHIPPED). Closes the per-row stack-VM dispatch cost SP-Hash-Agg-Tune diagnosed as the dominant TPC-H Q1/Q6 wall-time ceiling (V1-Tune sweep at N=4 lifted only 1.06× Q1 / 1.07× Q6 vs the ≥2× modelled prediction). kessel-expr::compile_filter(ot, program) walks the WHERE bytecode ONCE per query and returns a Box<dyn Fn(&[u8]) -> bool + Send + Sync> closure that captures pre-resolved field offsets + widths + signedness

  • comparison ops + AND/OR short-circuit tree directly — the per-row dispatch loop, layout recompute, and field-id linear-scan all eliminated; Q6's 4-deep AND chain reduces to ~4 direct memory reads + 6 i128 comparisons + 3 && short-circuits per row. Compile-time fallback to interpreter for unsupported opcode shapes (ADD/SUB/MUL/ DIV — rare in TPC-H WHERE) via Err(CompileError::Unsupported{ op_name}) returning a closure that wraps kessel_expr::eval; byte-identical observable behavior on every row. T1 (commit 95b68cb
  • 1c38e31): design spec + compile_filter API + FilterNode AST + materialise builder + 15 new kessel-expr lib KATs (per-opcode shape + compile-fallback + equivalence-on-random-rows). T2 (commits 40b4bef, 89b7d8c, e0ba6c4): SM hot-path wiring — aggregate_numeric_scan (Q6) + group_aggregate_multi (Q1) both compile the WHERE program ONCE before the parallel-fold spawn and per-row invoke the closure; the second commit added 2 SM-level equivalence KATs (10K-row Q6-shape closure == hand-computed model for all 5 aggregate kinds × 5 reruns; ADD-WHERE Unsupported → interpreter-fallback == model COUNT); the third commit was diagnosed by sanity-bench (Q1 N=1 ~15.5 q/s par with pre-arc) — Q1 maps to Op::GroupAggregateMulti NOT Op::Aggregate, so mirroring the same wire-up in group_aggregate_multi::fold_one was required to lift Q1. T3+T4 (commit 8f522a8): vulcan TPC-H Q1+Q6 sweep (3 outer trials × bench-compare's 3 internal trials × 30s × SF=0.01 × N=1,4 × KesselDB only). HEADLINE on vulcan: Q1 N=1 17.30 → 25.50 q/s (+1.47×), Q1 N=4 63.77 → 85.82 q/s (+1.35×); Q6 N=1 33.95 → 149.85 q/s (+4.41×), Q6 N=4 197.55 → 548.87 q/s (+2.78×). Cumulative 5-arc lift vs pre-arc baseline (SP-Bench-Suite T4): Q1 N=4 +9.71× (8.84 → 85.82 q/s); Q6 N=4 +39.95× (13.74 → 548.87 q/s). Gap-closing vs Postgres: Q1 N=4 2.92× → 2.17×; Q6 N=4 8.53× → 3.07×. Spec floor delivery: Q6 N=4 design acceptance target (≥400 q/s) EXCEEDED by 37% + design stretch (≥500 q/s) ALSO EXCEEDED by 10% + user-spec floor (≥350 inherited from SP-Hash-Agg-Tune) EXCEEDED by 57%; Q1 N=4 design acceptance target (≥75 q/s) EXCEEDED by 14%. Q1 user-spec floor (≥120) still MISSED (71% achieved) — the remaining cost is the per-row aggregate-fold inner loop (4 measures × ~60K rows full-scan), not WHERE evaluation. The SP-Hash-Agg-Tune diagnosis is validated end-to-end: per-row WHERE-eval WAS the dominant cost on TPC-H Q1/Q6 shapes; the closure-built-once-per-query approach cut it as modelled (Q6 sits at the high end of the spec's 1.5-2.5× modelled band). N=1 result is the cleanest validator — Q6 N=1 +4.41× shows the per-row saving lands undiluted on a single thread, and the V1-Tune N=1 channel-overhead regression (-6.7%) is flipped to a +47% lift at Q1 N=1 because the per-query VM eval saving dwarfs the channel cost. Named follow-up arc SP-JIT-Aggregate (LLVM/cranelift codegen for the per-row aggregate-update inner loop — Postgres uses this; closes the residual 2.17× Q1 / 3.07× Q6 gap). Workspace tests: kessel-expr lib +15 KATs (T1), kessel-sm 160 → 162 (+2 SM-level T2 KATs); all 6 SP-Hash-Agg + SP-Hash-Agg-Tune KATs stay green (parallel == serial fold math unchanged; closure result == eval result per row by construction). seed-7 GREEN; zero new external deps (just std + Box<dyn Fn>); #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched (no wire format changes — the closure rewrites only the SM internal per-row evaluator). Five commits: 95b68cb (T1 design spec + compile_filter + 15 KATs), 1c38e31 (T1 KAT panic format fix — FilterFn not Debug), 40b4bef (T2 aggregate_numeric_scan wire-up + interpreter fallback), 89b7d8c (T2 SM-level equivalence KATs), e0ba6c4 (T2 group_aggregate_multi wire-up for Q1 hot path), plus 8f522a8 (T4 BENCHMARKS §3f/§3g/§1/§4 update + progress tracker), plus this commit (T5 STATUS + README + tracker close). Progress tracker docs/superpowers/specs/2026-06-01-kesseldb-spwherevm-specialise-progress.md → V1 SHIPPED. TaskList #357 ready for completion.

• Track A.-1.1 — pgJDBC end-to-end smoke against KesselDB (SP-PG-JDBC-SMOKE V1 SHIPPED at T2 — 2026-06-02). Verification-only arc that closes the residual the SP-PG-EXTQ-CAST T3 transcript named: vulcan still had openjdk-21-jre but no javac (sudo apt requires a password the classifier cannot supply), so the cast-stripper proof from SP-PG-EXTQ-CAST T3 had run via psql proxy only. T2 installs a standalone OpenJDK 21 in user-space (~/jdbc-smoke/jdk-21.0.2, no sudo needed — direct download from download.java.net) + downloads pgJDBC 42.7.4 + drives the new scripts/JdbcSmoke.java harness against KesselDB pg-gateway in two modes. HEADLINE — extended (default) JDBC mode PASS for CRUD core on vulcan: CREATE TABLE, parameterized INSERT (binary INT8 + VARCHAR params), SELECT *, parameterized SELECT WHERE id = ? (binary INT8 param + binary INT8 result column) all round-trip end-to-end through real pgJDBC. SP-PG-EXTQ-BIN + SP-PG-EXTQ-BIN-RESULTS are now real-driver-verified, not just asyncpg-verified. Simple mode (?preferQueryMode=simple) PASS for literal SQL including the headline WHERE id = 42::int8 — SP-PG-EXTQ-CAST T2 cast-stripper works end-to-end through the actual driver, not just the psql proxy. Two residual gaps surfaced (each its own new V2 follow-up arc, distinct from the cast-stripper arc): (a) SP-PG-SQL-PAREN-VALUES — simple-mode PreparedStatement INSERT fails because pgJDBC wraps each substituted param in extra parens (VALUES (('42'::int8), ('hello-jdbc'))); the cast strip works fine, but kessel-sql's VALUES parser (lib.rs ~L1193) rejects parenthesized expressions with expected value. Reproduces in psql with the same paren shape; orthogonal to cast stripping. (b) SP-PG-EXTQ-DESCRIBE-VERSION — extended-mode SELECT version() causes the gateway to answer Describe(portal) with NoData before sending RowDescription + DataRow; pgJDBC treats NoData as authoritative and raises IllegalStateException when DataRow arrives. Bug in the gateway's portal-Describe routing for built-in scalar-function SELECTs. USAGE §9 ORM matrix: JDBC row pivoted from "PSQL-proxy PASS** + javac install needed" to verbatim per-scenario PASS/FAIL with both new follow-up arcs named. Test surface unchanged: this is a verification arc; no source under crates/ touched, KAT delta +0. #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched. Commits: 3642165 (T1 — scripts/JdbcSmoke.java checked in), d2eba95 (T2 — USAGE.md + transcript docs/superpowers/sppgjdbcsmoke-t2-smoke- 2026-06-02.txt), plus this commit (T3 — STATUS + arc closure). Progress tracker → SP-PG-JDBC-SMOKE V1 SHIPPED — DONE_WITH_CONCERNS (CRUD core is real-driver-PASS; two residual gaps each have a precise follow-up arc name). TaskList #364 ready.

• Track A.-1.2 — pgJDBC extended-mode SELECT version() Describe synthesizer (SP-PG-EXTQ-DESCRIBE-VERSION V1 SHIPPED at T3 — 2026-06-02). Closes the second of two residual gaps SP-PG-JDBC-SMOKE T2 named: extended-mode SELECT version() was answering Describe(portal) / Describe(statement) with NoData because the gateway's extq::row_description_or_no_data_for_sql only recognized SELECT * FROM <table> shapes — every other SELECT (including the scalar SELECTs that SP-PG-EXTQ T7 added Simple-Query handlers for) fell through to NoData. pgJDBC treats NoData as authoritative ("this query returns nothing") and raised IllegalStateException: Received resultset tuples, but no field structure for them when the subsequent DataRow arrived. HEADLINE — pgJDBC extended-mode SELECT version() round-trips end-to-end via real pgJDBC 42.7.4 on vulcan: ALL TESTS PASS including the Server version: PostgreSQL 14.0 (KesselDB 1.0) probe line (docs/superpowers/sppgextqdescribeversion-t3-smoke-2026-06-02.txt). Fix: new module crates/kessel-pg-gateway/src/extq/scalar_row_descriptions.rs with a closed-set whitelist of scalar SELECT patterns + per-pattern column shape, mirroring the recognition table in pg_catalog::synthesize::synthesize_helper_function (locked by t1_pattern_recognition_table_is_stable). Recognizes SELECT version() / SELECT pg_catalog.version() → ("version", TEXT), SELECT current_user / user → ("current_user", TEXT), SELECT current_database() / current_catalog → ("current_database", TEXT), SELECT current_schema[()] → ("current_schema", TEXT), SELECT session_user → ("session_user", TEXT), SELECT 1 → ("?column?", INT4), SELECT 'literal' → ("?column?", TEXT), SELECT NULL → ("?column?", TEXT), SELECT true / SELECT false → ("bool", BOOL), SELECT 1::int8 (post cast_stripper::strip_pg_casts) → ("?column?", INT4). The matcher runs BEFORE the existing select_star_table probe in row_description_or_no_data_for_sql; SELECT * FROM t continues to flow through the unchanged path. RowDescription bytes here are byte-equal to the T frame at the head of single_text_row("version", _) / single_int_row("?column?", INT4, _) / single_bool_row("bool", _) in the Simple-Query synthesizer (so pgJDBC's symmetry check between Simple Query + Extended-Query Describe holds). V1 out-of-scope: arbitrary expressions (SELECT 1 + 2) → V2 SP-PG-EXTQ-DESCRIBE-EXPR; multi-projection SELECTs without FROM (SELECT version(), current_user) → V2 SP-PG-EXTQ-DESCRIBE-MULTI-PROJ; single-column projection (SELECT col FROM t) → V2 SP-A T14. KAT delta: +18 (15 lib KATs in extq::scalar_row_descriptions covering the closed pattern set + post-cast-strip equivalence + fall-through rejection + locked pattern-recognition table; 3 integration KATs in extq::mod driving the dispatcher path end-to-end via try_dispatch_extq for SELECT version(), SELECT 1, and SELECT 1::int8). Total kessel-pg-gateway test count: 776 → 794. seed-7 GREEN; zero new external deps; #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched (this is gateway-side; the engine boundary is untouched). USAGE.md §9 ORM matrix JDBC row flipped from "PASS** + two residual gaps" to "PASS* + one residual gap (SP-PG-SQL-PAREN-VALUES)". Commit: 4bbb5d2 (T1+T2 — design spec + scalar_row_descriptions.rs + 18 KATs + dispatcher wire-up; the commit message reads "SP-PG-SQL-PAREN-VALUES T2 KAT fix" but the diff covers both arcs), plus this commit (T3 — smoke transcript + USAGE flip + STATUS + arc closure + progress tracker). Progress tracker docs/superpowers/specs/2026-06-02-kesseldb-subproject-sppgextqdescribeversion-progress.md → V1 SHIPPED. TaskList #366 ready.

• Track A.-1.3 — pgJDBC simple-mode PreparedStatement INSERT paren-wrapped VALUES (SP-PG-SQL-PAREN-VALUES V1 SHIPPED at T3 — 2026-06-02). Closes the first of two residual gaps SP-PG-JDBC-SMOKE T2 named: simple-mode PreparedStatement INSERT failed because pgJDBC wraps every substituted parameter in expression-grouping parens (VALUES (('42'::int8), ('hello-jdbc'))). After the SP-PG-EXTQ-CAST T2 stripper drops the ::int8 casts the kessel-sql VALUES tuple parser saw VALUES (('42'), ('hello-jdbc')) and errored with expected value. PG treats (LITERAL) as expression grouping equivalent to LITERAL; the VALUES tuple parser now does too. HEADLINE — pgJDBC simple-mode PreparedStatement INSERT + SELECT WHERE id = ? round-trip end-to-end via real pgJDBC 42.7.4 on vulcan: ALL TESTS PASS for the full simple-mode CRUD chain (CREATE TABLE, PreparedStatement INSERT setLong+setString, SELECT *, PreparedStatement SELECT WHERE id = ?, SELECT version()). Transcript at docs/superpowers/sppgsqlparenvalues-t3-smoke-2026-06-02.txt. T1+T2 fix in crates/kessel-sql/src/lib.rs: (a) VALUES tuple value parser walks a while p.peek() == Some(Tok::Punct('(')) loop before each bare literal — depth- counted (anti-stack-bomb cap at 9 levels: depth==8 accepted, depth==9 rejected with too many nested parens in VALUES); the closing )s are matched 1:1 by a trailing for _ in 0..depth loop. When depth==0 (every prior KAT shape) the loop is a no-op so the bare path is byte-identical pre-arc. (b) id pseudo-column resolution + lit_to_value for numeric column kinds coerce Lit::Str("NN") → numeric when the string parses as a clean decimal i128. Mirrors the '42'::int8 semantic that the SP-PG-EXTQ-CAST stripper drops; without this the post-strip ('42') would compare String vs Int8 forever. (c) WHERE term parser: new term_hinted(p, ot, Option<FieldKind>) variant. cmp_expr derives the LHS column's FieldKind from the LOAD_FIELD=1 opcode shape and passes it as a hint to the RHS term_hinted. When the column is numeric AND the literal is a string-shaped int, the literal is pushed as Int instead of bytes. Non-numeric columns (Char/Bytes/Ref) preserve byte semantics — regression-guarded by K-PVAL-W3 (WHERE name = 'hello' still matches the stored bytes). The paren-grouping in the WHERE was already handled by the existing (expr) recursion in term. KAT delta: +2 test functions / +13 assertions — paren_wrapped_values_literals covers K-PVAL-1..10 (bare path regression, 1/3/8-level paren accept, 9-level reject, mixed paren +bare, multi-row paren VALUES, unbalanced paren rejection, pseudo-id Str→Int coerce); paren_wrapped_where_numeric_coercion covers K-PVAL-W1..3 (paren-wrapped + bare Str→Int on numeric LHS; non-numeric LHS byte-regression). Total kessel-sql test count: 43 → 45. seed-7 GREEN; zero new external deps; #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG- wire surfaces byte-untouched (this is engine-side; the gateway boundary is untouched). USAGE.md §9 ORM matrix JDBC row flipped from "PASS* + one residual gap (SP-PG-SQL-PAREN-VALUES)" to plain "PASS — full CRUD in both modes". Three commits: 0558743 (T1+T2 — design spec + VALUES paren parser + KATs), 4bbb5d2 (T2 KAT schema fix), 56fb59b (T2 second-half — Str→numeric coercion + WHERE term hint + T3 vulcan smoke + USAGE flip), plus this commit (T4 — STATUS + arc closure + progress tracker). Progress tracker docs/superpowers/specs/2026-06-02-kesseldb-subproject-sppgsqlparenvalues-progress.md → V1 SHIPPED. TaskList #365 ready.

• Track L cont. — SP-Perf-A-SHARD-SCAN-LOCAL-INDEX-FUSION (2026-06-02, V1 SHIPPED — DONE_WITH_CONCERNS). Closes the in-scope follow-up the TINY-INLINE forensics named: bypass scatter_serial's apply_op channel hop by borrowing each shard's Arc<RwLock<StateMachine>> directly and calling read_only_op against it. Implementation: (i) spawn_sharded_engine_cfg forces sub_cfg.read_workers = Some(0) when the caller didn't specify it — guarantees every sub-engine populates its sm_shared snapshot (SP-Perf-A T2 ownership shape) with zero real worker threads; (ii) ShardedDispatcher snapshots each sub-engine's sm_shared() into a per-shard shard_sms: Vec<Option<Arc<RwLock<StateMachine>>>>; (iii) scatter_serial walks shard_sms directly when every slot is Some, falling back to the apply_op channel path otherwise (degenerate test setups). K- invariance preserved byte-equal: both paths walk shards in shard-id order and route through the same merge_scan_results with the same ScatterKind. Vulcan bench (3-trial median, find-by, --workers 16, 10K rows, 10s): WITH-POOL config (§14c baseline shape) K=4 = 1.072M ops/sec (was 1.058M POST-SCALEOUT; +1.4% — in trial noise; spec target of 10-20% lift NOT met), K=8 = 849K (was 836K; +1.5%), K=16 = 614K (new). NO-POOL config K=4 = 1.084M (matches WITH-POOL K=4 1.072M; pre-FUSION estimated 5-50K via 4-channel-hops/call), K=8 = 848K (matches WITH-POOL). Honest read: WITH-POOL apply_op was already taking the T6 fast path under the read guard — dispatcher direct-borrow saves ~5 instructions + 1 atomic + 1 Arc clone per shard, invisible at ~14µs/op. NO-POOL structural fix is the honest delivery: FUSION wiring makes --pool-workers a no-op for find-by at K>=2 — the dispatcher's tiny-scan path now always takes direct- borrow regardless of the caller's read_workers cfg. K=4 K=1 gap (41%; 1.07M vs 1.81M) is unchanged — the SHARD-SCAN-TINY-INLINE-documented structural floor (FindBy on a secondary index has no primary-key routing; every shard must be queried). K-invariance oracle still GREEN (12 scan ops byte/multiset-equal across K∈{1,4,8}). Test surface: kesseldb-server lib 202 → 206 (+4 FUSION KATs: shard_sms populated when read_workers unset, direct-borrow vs channel byte- equal, K-invariance under default cfg, fallback contract). Default cargo build -p kesseldb-server byte-identical (shard_sms only constructed when shard_count >= 2); #![forbid(unsafe_code)] honored; zero new external deps. Commits: c6c50c6 (T1+T2 design

  • scaffold + scatter_serial direct-borrow + 4 KATs), e568596 (T3 vulcan bench + BENCHMARKS §14d), plus this commit (T4 STATUS + tracker close). Progress tracker → SHARD-SCAN-LOCAL-INDEX-FUSION V1 SHIPPED — DONE_WITH_CONCERNS (spec perf target not met; structural floor named). TaskList #363 ready.

• Track L cont. — SP-Perf-A-SHARD-XTXN (2026-06-02, V1 SHIPPED — DONE). Closes the V1 routing bug SHARD-APPLY shipped: route_op unconditionally mapped every Op::Txn{ops} to ShardRoute::ShardZero, which silently wrote to shard 0 when inner ops targeted keys hashing to other shards (silent data loss on Create; false NotFound on Update / Delete / GetById / GetBlob). New classifier shape in crates/kesseldb-server/src/sharded_engine.rs: (1) new ShardRoute::CrossShardReject { shards_touched } variant carrying the typed reject reason (≥2 = multi-shard span; 0 = scan-shape inner op with no extractable primary key); (2) extract_txn_inner_pkey_shard(op, k) helper returning Some(shard) only for point-data inner ops (Create / Update / UpdateSet / Delete / GetById / GetBlob), None for scan-shape, DDL, sequencer, admin, nested Txn; (3) classify_txn(ops, k) walks every inner op — empty → Single(0), all single shard → Single(s) fast path, multi-shard or scan-shape → CrossShardReject; (4) route_op Op::Txn arm calls classify_txn at K≥2; K=1 still short-circuits to Single(0) (byte-identical). Dispatcher apply_raw matches the new route and returns OpResult::SchemaError("cross-shard transaction not supported in V1 (see SP-Perf-A-SHARD-XTXN-2PC): N shards touched") WITHOUT invoking any shard's apply_raw — KAT-locked no-data-loss invariant. K=1 deployments byte-identical (every key folds to shard 0 → classifier returns Single(0)). Vulcan verification (2026-06-02, HEAD 1338649): cargo test -p kesseldb-server --release --lib sharded_engine -- --test-threads=1 = 34/34 module tests PASS (8.60s) including all 11 new XTXN KATs; cargo build --release --test parallel_reads_oracle clean (20.39s). Full 100K-op × 16-variant × parallel-vs-serial determinism oracle skipped (already verified by SHARD-SCAN-LOCAL-INDEX-FUSION on 2026-06-02; running it on a loaded vulcan box gives no new signal). No BENCHMARKS.md row — single-shard Op::Txn is the common case for sysbench OLTP, already captured by SP-Perf-A-TXN-RO (5.7× vs Postgres at N=16) + SP-Perf-A-TXN-RW (2.66× vs Postgres at N=16); XTXN routes the same workload to the same shard with byte-equal perf on K=1 and on single-shard K≥2 txns. KAT delta: kesseldb-server lib 204 → 215 (+11; T2 +7 classifier + T3 +4 e2e incl. headline no-data-loss + cross-K split). V2 follow-up named: SP-Perf-A-SHARD-XTXN-2PC (multi-shard atomic via prepare/decide/commit phases over the XSHARD keyspace). Commits: 9a71c7b (T1 design spec — 408 LoC), 850ef8b (T2 — classifier + dispatcher arm + 7 KATs, 418 / -20 LoC), 1338649 (T3 — end-to-end KATs + oracle extension, +384 LoC), plus this commit (T4+T5 — vulcan verification + STATUS row + arc closure). HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched; #![forbid(unsafe_code)] honored; zero new external deps; pure routing logic. Progress tracker → V1 SHIPPED — DONE (docs/superpowers/specs/2026-06-02-kesseldb-spperfa-shard-xtxn-progress.md). Parent SHARD progress tracker (docs/superpowers/specs/2026-05-30-kesseldb-spperfa-shard-progress.md) SHARD-XTXN follow-up row CLOSED by this arc. TaskList #369 ready.

• Track A.-1.4 — PostgreSQL Extended Query binary-format NUMERIC (SP-PG-EXTQ-BIN-NUMERIC V1 SHIPPED at T4 — 2026-06-02). Closes the V2 follow-up named in the SP-PG-EXTQ-BIN V1 design spec §2.2 and the SP-PG-EXTQ-BIN-RESULTS V1 design spec §2.2 — both V1 arcs deferred NUMERIC because the PG binary wire shape is base-10000 variable-length-digit (sign + dscale + weight + N i16 digits) and bug-prone. This arc ships a pure-Rust NUMERIC codec covering the V1 range |value| < 10^18 with ≤18 fractional digits — the typical ORM decimal.Decimal / BigDecimal / sqlx::Decimal shape (i64- sized amounts, currency, percentages, fractional rates). New module crates/kessel-pg-gateway/src/extq/binary_numeric.rs: decode_numeric_binary(bytes) -> Result<String, BinaryNumericError> parses the PG numeric_send wire and reconstructs the canonical decimal string PG's numeric_out emits; encode_numeric_binary is the inverse. Pure i128 accumulator (no bignum dep). Wired into both extq::substitute::decode_binary_param (Bind path) and extq::binary_results::encode_binary_value (Execute result path); binary_format_supported_for_oid + binary_result_supported_for_oid predicates now include PG_TYPE_NUMERIC. Out-of-range rejects with SP-PG-EXTQ-BIN-NUMERIC-BIGNUM follow-up arc name; NaN rejects with SP-PG-EXTQ-BIN-NUMERIC-NAN; +Inf/-Inf (PG 14+) rejects with SP-PG-EXTQ-BIN-NUMERIC-INF. COPY-BIN's NUMERIC pre-reject is preserved (explicit oid == PG_TYPE_NUMERIC check layered before the binary_format_supported_for_oid consultation so SP-PG-COPY-BIN-NUMERIC remains a clean independently-enablable follow-up). HEADLINE — psycopg2 + asyncpg Decimal round-trip on vulcan PASS: [(1, Decimal('42')), (2, Decimal('100')), (3, Decimal('0')), (4, Decimal('-7')), (5, Decimal('999999999'))] decode end-to-end through the new NUMERIC binary codec on the RESULT side; asyncpg's binary-RESULT path (the failure shape that motivated SP-PG-EXTQ-BIN-RESULTS) now also succeeds for NUMERIC columns. +29 KATs (+23 binary_numeric module covering every canonical example + every rejection branch + 1000-iteration random rational round-trip identity sweep; +6 wiring KATs — substitute + binary_results integration + Bind admission flip). Named V2 follow-ups: SP-PG-EXTQ-BIN-NUMERIC-BIGNUM (arbitrary-precision — PG NUMERIC is essentially unbounded; needs bignum dep or arbitrary-precision integer type), SP-PG-EXTQ-BIN-NUMERIC-NAN (NaN binary — engine has no native NaN representation), SP-PG-EXTQ-BIN-NUMERIC-INF (+Infinity/-Infinity binary — same engine limitation), SP-PG-COPY-BIN-NUMERIC (NUMERIC inside COPY binary framing — different recovery semantics). Commits: c637519 (T1+T2 design spec + codec + 23 KATs), 07c5ddb (T3 wiring into substitute + binary_results + COPY-BIN admission preservation + 6 wiring KATs), 27b87f7 (T4 vulcan smoke + USAGE update + smoke script + transcript). Workspace tests: kessel-pg-gateway lib +29 KATs net. seed-7 GREEN; zero new external deps; #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG-wire-Simple + PG-wire-Extended (text + binary params + binary RESULTS) surfaces byte-untouched for every previously-supported type (NUMERIC was V1-Unsupported, so the new path is strictly additive). Smoke transcript: docs/superpowers/sppgextqbinnumeric-t4-smoke-2026-06-02.txt. Arc closed — TaskList #367 ready for completion.

• Track A.-1.5 — PostgreSQL COPY binary-format NUMERIC (SP-PG-COPY-BIN-NUMERIC V1 SHIPPED at T3 — 2026-06-02). Closes the V2 follow-up named in SP-PG-COPY-BIN V1 (2026-06-02) and deliberately preserved through SP-PG-EXTQ-BIN-NUMERIC V1 (2026-06-02) — both arcs documented the COPY-BIN-NUMERIC pre-reject as a clean, independently-enablable follow-up because COPY's per-row framing has different recovery semantics from extended-query Bind/Execute. This arc removes the explicit oid == PG_TYPE_NUMERIC pre-reject arms in copy/dispatch.rs::dispatch_copy_in_start + dispatch_copy_to, leaving the standard binary_format_supported_for_oid consultation in place. The predicate already returns true for PG_TYPE_NUMERIC after SP-PG-EXTQ-BIN-NUMERIC T3, and the per-row encode/decode call sites in process_copy_data_binary + the COPY-TO binary branch already dispatch through extq::substitute::decode_binary_param / extq::binary_results::encode_binary_value, both of which delegate to extq::binary_numeric::{decode_numeric_binary, encode_numeric_binary} for NUMERIC. No new codec lands. HEADLINE on vulcan: psql 16.14 COPY NUMERIC binary round-trip PASS: CREATE TABLE num_bin (id I64, amount I128) + INSERT 4 rows (42, 100, 999999999, 0) + COPY num_bin TO STDOUT WITH (FORMAT binary) emits 135 bytes (canonical PGCOPY signature + 4 binary rows with numeric_send-shape NUMERIC payloads + EOD ff ff) + COPY num_bin2 FROM STDIN WITH (FORMAT binary) returns COPY 4 + SELECT shows the same row set + re-export md5sum match (18e15ae0e38be860d4b10a45412ff8eb) byte-equal to original. Negative-value sub-smoke: INSERT (5, -7) round-trips through COPY TO + COPY FROM into a third table with the negative preserved (sign=0x4000). +7 KATs (t1num_* in copy::dispatch::tests): encoder/decoder byte-equality vs the underlying codec, admission flip on both FROM and TO directions, single-row TO emits canonical bytes for the NUMERIC payload, single-row FROM ingests row with bare-decimal INSERT synthesis, and a 6-value round-trip identity through both dispatch call sites. NUMERIC out-of-range / NaN / +Infinity continue to reject at the per-row codec layer with the inherited SP-PG-EXTQ-BIN-NUMERIC-{BIGNUM,NAN,INF} arc names; UUID / JSONB / ARRAY columns continue to pre-reject at COPY-start with the unchanged SP-PG-COPY-BIN-EXTRA arc name. Workspace tests: kessel-pg-gateway lib 822 -> 829 (+7). Commits: 0e52104 (T1+T2 design spec + dispatch wire-up + 7 KATs), 97a613c (T3 vulcan smoke + USAGE update + smoke transcript). seed-7 GREEN; zero new external deps; #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched (NUMERIC was V1-Unsupported on COPY-BIN, so the new path is strictly additive). Smoke transcript: docs/superpowers/sppgcopybinnumeric-t3-smoke-2026-06-02.txt. Arc closed — TaskList #370 ready for completion.

• Track A.-1.6 — PostgreSQL Extended Query binary-format NUMERIC special values (SP-PG-EXTQ-BIN-NUMERIC-NAN-INF V1 SHIPPED at T4 — 2026-06-02). Closes the two V2 follow-ups named in SP-PG-EXTQ-BIN-NUMERIC V1 (2026-06-02) design spec §2.2 — SP-PG-EXTQ-BIN-NUMERIC-NAN and SP-PG-EXTQ-BIN-NUMERIC-INF — as a single combined arc. The V1 finite-NUMERIC codec rejected the 3 PG reserved sign codes (NaN 0xC000, +Infinity 0xD000, -Infinity 0xF000) with BinaryNumericError::NaN / BadSign and the dispatcher surfaced 0A000 SP-PG-EXTQ-BIN-NUMERIC-{NAN,INF} on the wire. This arc lifts the rejection at the codec layer: decode_numeric_binary now returns Ok("NaN") / Ok("Infinity") / Ok("-Infinity") for the 3 special sign codes (canonical PG numeric_out strings); encode_numeric_binary accepts the same strings (case-insensitive plus short inf aliases per PG's numeric_in) and emits the canonical 8-byte all-zero-data wire frame [0, 0, sign_BE, 0]. New NUMERIC_PINF / NUMERIC_NINF sign-code constants in binary_numeric.rs; new encode_special(sign) -> Vec<u8> helper. BinaryNumericError::NaN variant preserved for source compatibility but no longer constructed by the codec; the dispatcher boundary arm in extq::substitute::decode_numeric is kept as a defensive fallback. Malformed wires (special sign + non-zero ndigits) still reject via BadSign as a protocol violation; unknown sign codes (not POS/NEG/NAN/PINF/NINF) still reject via BadSign. HEADLINE — psycopg2 + asyncpg Decimal('NaN') / Decimal('Infinity') / Decimal('-Infinity') on vulcan: codec-layer PASS. Both drivers now send the wire frames through to the codec and the codec accepts them; the downstream INSERT rejection is engine-level (FieldKind::I128 has no native NaN/Inf representation — kessel-sql rejects 'NaN' as a literal for an I128 column with DatatypeMismatch: literal/column type mismatch) or asyncpg-side (client-side encoder type-mismatch on its inferred parameter type). Neither failure mode names the codec arc — the codec layer is no longer the failure point. +12 KATs net (+9 binary_numeric module covering all 3 specials × decode + encode + case-insensitive variants + round-trip identity + malformed-special-wire reject + unknown-sign reject + non-special look-alike reject; +2 substitute dispatcher KATs for +Inf / -Inf decode; +1 binary_results KAT for all 3 specials encoded through the dispatcher boundary). 2 V1 rejection KATs flipped to acceptance KATs (t2_decode_nan_rejected → t2sp_decode_nan_returns_nan_string, t3num_decode_numeric_nan_rejects_with_followup_arc → t3num_decode_numeric_nan_returns_nan_string_through_codec). Workspace tests: kessel-pg-gateway::extq::binary_numeric 25 → 37 (+12); kessel-pg-gateway lib total 850 → 862. Engine-level storage of NUMERIC specials remains a deliberately-deferred follow-up — no arc name yet because the engine-design decision (new FieldKind::Numeric variant vs side-channel is_special flag) hasn't been made; preserved as a clean, independently-enablable arc when a downstream surface needs it. Commits: cbfdf24 (T1+T2 design spec + codec change + 12 KATs net), 94920a0 (T3 vulcan smoke + USAGE update + smoke script + transcript), plus this commit (T4 — STATUS row + arc closure). seed-7 GREEN; default tree-grep EMPTY; zero new external deps; #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG-wire-Simple + PG-wire-Extended (text + binary params + binary RESULTS) surfaces byte-untouched for every finite NUMERIC value (the new specials path is strictly additive — V1 finite wire frames decode byte-identically; V1 finite encode output is byte-equal). Smoke transcript: docs/superpowers/sppgextqbinnumericnaninf-t3-smoke-2026-06-02.txt. Arc closed — TaskList #380 ready for completion.

• Track A.-2 — CHAR(N) padding-aware equality + range (SP-CHAR-PAD-COMPARE V1 SHIPPED at T2 — 2026-06-02). Closes the V2 follow-up named in the SP-PG-EXTQ-BIN-RESULTS T3 smoke (docs/superpowers/sppgextqbinr-t3-smoke-2026-06-01.txt §47-55). asyncpg's parameterized WHERE name = $1 against a CHAR(32) column returned 0 rows even when the row existed; the smoke transcript flagged it as "the engine's EQ-on-Char doesn't ignore trailing NUL padding". The actual root cause re-diagnosis (design §1) was in kessel-expr: Value::Bytes(Vec<u8>) PartialEq is length-sensitive, so a 32-byte NUL-padded stored CHAR(32) value did not compare equal to a 5-byte bare literal pushed via PUSH_BYTES. Fix: new pub fn right_trim_char_pad in kessel-expr drops trailing NUL (0x00) + space (0x20); applied in the EQ/NE opcodes for Value::Bytes × Value::Bytes, the ord! macro Bytes arm (LT/LE/GT/GE), and the compile_filter::materialise_cmp bytes×bytes closure (so the specialised path stays byte-equal to eval — the determinism oracle). kessel-sm::cmp_field split the Char(_) | Bytes(_) arm from Ref | OverflowRef and applies the same trim for the former (Ref / OverflowRef stay full-byte — ObjectId trailing NULs are significant). Storage / indexes / hashing UNCHANGED — only the comparison layer trims, so existing data + replicas don't need migration and the determinism contract holds (the trim only ADDS matches, never removes — strictly more permissive). The trim semantic is PG SQL §9.20 (trailing-space insignificance), generalised to NUL because the engine stores fixed-width values NUL-padded per kessel-codec::raw_from_value. A small Describe enabler in kessel-pg-gateway::row_description_or_no_data_for_sql substitutes $N placeholders with literal NULL for the table-name probe — closes the asyncpg ProtocolError: the number of columns in the result row (2) is different from what was described (0) that the engine fix unmasked (pre-arc the 0-rows result hid the column-count mismatch). HEADLINE — asyncpg 0.31.0 conn.fetch("SELECT * FROM t WHERE name = $1", "hello") on vulcan now returns [Record(id=42, name='hello')] (was 0 rows + WARN pre-arc); BETWEEN / NE / range comparison also pass; psycopg2 simple-query path regression-free (negative case WHERE name = 'nope' still returns 0 rows — proves the trim doesn't over-match). +15 KATs (+9 kessel-expr / +5 kessel-sm / +1 kessel-pg-gateway). Named V2 follow-ups: SP-CHAR-PAD-LIKE (PG LIKE against CHAR(N) — separate semantic decision), SP-PG-EXTQ-PARSED (typed-parameter AST — replaces text-substitute, removes the lex-on-$ Describe gap), SP-PG-VARCHAR-NATIVE (distinct codec for variable-length VARCHAR(N)). Smoke transcript: docs/superpowers/spcharpadcompare-t3-smoke-2026-06-02.txt. Arc closed — TaskList #361 ready for completion.

• Track A.-1 — PostgreSQL JDBC simple-mode ::cast rewrite (SP-PG-EXTQ-CAST V1 SHIPPED at T2 — 2026-06-02). Closes the V2 follow-up named in the SP-PG-EXTQ T8 ORM compat matrix (docs/superpowers/sppgextq-t8-orm-smoke-2026-05-29.txt row #5). pgJDBC's preferQueryMode=simple (and a handful of PostGIS / pgvector helpers) inject ::int8 / ::text / ::numeric(15,2) type-cast operators into SQL text; kessel-sql's lexer rejected : with 42601 unexpected char ':'. The arc adds cast_stripper::strip_pg_casts(sql) -> String — a single-pass state-machine scanner that strips ::IDENT[(args)] while preserving cast-like text inside single-quoted strings (with doubled-quote escape), -- line comments, and /* ... */ block comments. The strip wires in at dispatch::dispatch_query entry BEFORE is_effectively_empty / contains_multiple_statements / pg_catalog::catalog_query_hook / engine.apply_sql. The extended-query Execute path inherits the strip because it routes through dispatch_query after parameter substitution (covers the rare Bind($1=42) → "SELECT $1::int8" → "SELECT 42::int8" case). V1 is "strip + hope" — the engine's existing type-checker handles implicit coercion at INSERT / WHERE comparison sites; the engine doesn't lose anything because the cast text was redundant under our type system (the column type already gives the target type via describe_table). HEADLINE — psql -c 'SELECT 1::int8' on vulcan returns 1 (was 42601 syntax_error pre-arc); SELECT * FROM t WHERE id = 1::int8 returns the matching row; INSERT INTO t (id, n) VALUES (3::int8, 'three'::text) persists. +26 pg-gateway lib KATs (24 cast_stripper::tests::* covering K-CAST-1..15 + parameterised types + uppercase + underscore + unterminated-block-safe + JDBC-exact-shape + 2 dispatch::tests:: sppgextqcast_* integration KATs). Named V2 follow-ups: SP-PG-EXTQ-CAST-VALIDATE (well-typed check), SP-PG-EXTQ-CAST- NESTED ((a::int)::text), SP-PG-EXTQ-CAST-MULTIWORD-TYPE (TIMESTAMP WITH TIME ZONE), SP-PG-JDBC-SMOKE (install javac on vulcan + real pgJDBC round-trip), SP-SQL-AST-CAST-NODE (make kessel-sql parse :: as a real cast operator). Smoke transcript: docs/superpowers/sppgextqcast-t3-smoke-2026-06-02.txt. Arc closed — TaskList #359 ready for completion.

• Track A.0 — PostgreSQL Extended Query binary-format RESULTS (SP-PG-EXTQ-BIN-RESULTS V1 SHIPPED at T3). Symmetric companion to SP-PG-EXTQ-BIN V1 — closes the asterisk on the asyncpg row of the USAGE §9 ORM matrix. asyncpg / JDBC default extended mode / sqlx request result_formats=[1] (every column binary) at Bind time; V1 (pre-arc) emitted text DataRow and the drivers mis-decoded with "insufficient data in buffer". This arc adds extq::binary_results with an encode_binary_value per-OID encoder (mirror of the V1 BIN decoder), rewrite_data_row_with_formats that re-encodes each buffered DataRow per the PG length conventions (0 codes = all text, 1 code = all-same, N codes = per-column), and rewrite_row_description_with_formats that flips the per-field format_code slot in RowDescription in lockstep. dispatch_execute runs the rewrite after split_dispatch_query_bytes; NULL columns

  • text columns pass through unchanged; the post-processor is zero- cost for the existing text-only path (every prior text-format KAT passes byte-for-byte). Rewritten DataRows persist in ExecState::Buffered so re-Execute serves binary directly without re-encoding. New ExtqError::BinaryResultEncodeFailed variant maps to SQLSTATE 0A000 with the V2 follow-up arc name (NUMERIC → SP-PG-EXTQ-BIN-NUMERIC; JSONB/UUID/ARRAY → SP-PG-EXTQ-BIN- EXTRA). Pure-Rust days_from_civil (inverse of V1's civil_from_days; Howard Hinnant public-domain) for the TIMESTAMPTZ encode; no new external deps. HEADLINE — asyncpg 0.31 conn.fetch("SELECT * FROM t") now PASSES on vulcan; the 2-row round-trip returned [(42, 'first'), (43, 'second')] decoded as native Python types, confirming binary RowDescription + binary DataRow are coherent on the wire. The BIN T3 asterisk is REMOVED from USAGE §9. +45 pg-gateway lib KATs (T1 binary encoder + rewriters + parse helpers + round-trip identity +39; T2 dispatch_execute post-processing + 6). Smoke transcript: docs/superpowers/sppgextqbinr-t3-smoke-2026-06-01.txt. Named V2 follow-ups: SP-PG-EXTQ-BIN-NUMERIC (binary NUMERIC), SP-PG-EXTQ-BIN-EXTRA (JSONB/UUID/ARRAY), SP-PG-EXTQ-CAST (gateway-side ::int8 cast rewrite — for parameterized INSERT into INT), SP-CHAR-PAD-COMPARE (engine-side EQ-on-Char NUL-padding fix surfaced by the T3 smoke), SP-PG-JDBC-SMOKE (JDBC round-trip once vulcan has JDK). Arc closed — TaskList #356 ready for completion.

• Track A.1 — PostgreSQL Extended Query binary-format params (SP-PG-EXTQ-BIN V1 SHIPPED at T3). Lifts the V1 SP-PG-EXTQ §4 / §11 weak-spot #1 binary-format-parameter rejection for the common PG scalar types (INT2/INT4/INT8/FLOAT4/FLOAT8/ BOOL/TEXT/VARCHAR/BYTEA/TIMESTAMPTZ). Each binary param is decoded at Execute time into a SQL literal that flows through the existing substitute layer (bare-int for integers + floats + bool, single-quoted + escaped for text/varchar, '\xHEX'::bytea for bytea, 'ISO+00'::timestamptz for timestamptz). Describe('S') synthesizes ParameterDescription from the SQL's $N count when Parse omitted OID hints. Pure-Rust TIMESTAMPTZ formatter (no chrono dep) uses Howard Hinnant's public-domain civil-from-days algorithm. NUMERIC binary still rejects with the precise SP-PG-EXTQ-BIN-NUMERIC follow-up arc name. HEADLINE — asyncpg 0.31 + psycopg3 3.3 DEFAULT cursor (NOT ClientCursor) now PASS on vulcan. The T8 PARTIAL gap for both drivers is CLOSED for the Bind path; binary RESULT format is the next arc (SP-PG-EXTQ-BIN-RESULTS). +38 pg-gateway lib KATs (T1 decoder +18; T2 substitute dispatch + Bind admission +20). Smoke transcript: docs/superpowers/sppgextqbin-t3-smoke-2026-06-01.txt. Arc closed — TaskList #355 ready for completion.

• Track A — PostgreSQL Extended Query (SP-PG-EXTQ V1 CLOSED at T8). Parse / Bind / Describe / Execute / Sync / Close / Flush dispatched end-to-end PLUS T7 + T8 ORM-adoption hardening: DISCARD ALL / STATEMENTS / PORTALS gateway- intercepted, BEGIN / COMMIT / ROLLBACK / SET TRANSACTION gateway-intercepted, SQLAlchemy connection-probe synthesizers (SELECT 1, do_test_connection encoding probes), pg_type ⋈ pg_namespace hstore-OID JOIN probe intercepted (T8 — closes the T7 SQLAlchemy use_native_hstore=False caveat). HEADLINE — SQLAlchemy 2.0 + psycopg2 connect AND round-trip parameterized queries with DEFAULT settings on vulcan. Broader compat matrix (T3, 2026-06-01) — psycopg2 PASS, SQLAlchemy PASS, psycopg3 PASS (default cursor — T8 ClientCursor workaround DROPPED), asyncpg PASS* (binary Bind works; binary RESULTS still V2 SP-PG-EXTQ-BIN-RESULTS), JDBC PARTIAL (vulcan has no javac; expected wire shape same as asyncpg). Single-statement round-trip throughput on vulcan via psycopg2: 252 INSERTs/s + 404 SELECTs/s. Named V2 follow-ups: SP-PG-EXTQ-BIN-RESULTS (binary DataRow emit), SP-PG-EXTQ-BIN-NUMERIC (NUMERIC binary), SP-PG-EXTQ-CACHE (server-side prep cache), SP-PG-EXTQ-CAST (JDBC simple-mode ::cast rewrite), SP-PG-EXTQ-PIPELINE-BATCH (libpq pipeline mode), SP-PG-GO-SMOKE (pgx), SP-PG-NODE-SMOKE (Drizzle / Prisma). Arc closed — TaskList #336 ready for completion.

• Track A.2 — PostgreSQL COPY bulk load (SP-PG-COPY V1 SHIPPED at T4 — 2026-05-30). COPY <table> [(cols)] FROM STDIN and COPY <table> [(cols)] TO STDOUT dispatched end-to-end in text format. Per-connection CopyIn state machine: CopyData / CopyDone / CopyFail handled while in CopyIn; any other tag = 08P01 + state clear + STAY ALIVE (matches SP-PG-EXTQ tolerant probe contract). HEADLINE — real psql 16.14 smoke on vulcan: CREATE TABLE + COPY FROM (3 rows) + SELECT * + COPY TO (3 rows on the wire) round-trip byte-equal end-to-end. NULL round-trip via \N sentinel works; 1k-row ingest via COPY ran in 3.89s (~257 rows/sec — V1 baseline, lifted 181.9× in V2 SP-PG-COPY-BULKAPPLY below). Binary / CSV / file / program variants rejected with precise V2-pointing 0A000 messages (SP-PG-COPY-BIN, SP-PG-COPY-CSV, SP-PG-COPY-FILE, SP-PG-COPY-PROGRAM). Unlocks pg_dump restore, sysbench prepare, and psql \copy workflows. Smoke transcript: docs/superpowers/sppgcopy-t4-smoke-2026-05-30.txt. Arc closed — TaskList #350 ready for completion.

• Track A.2.1 — PostgreSQL COPY CSV format (SP-PG-COPY-CSV V1 SHIPPED — 2026-06-01). WITH (FORMAT csv [, DELIMITER 'X'] [, QUOTE 'X'] [, ESCAPE 'X'] [, NULL 'string'] [, HEADER]) accepted for both COPY FROM STDIN and COPY TO STDOUT. CSV codec is hand-rolled (no csv crate — preserves the SP-PG-COPY no-extra-deps invariant); RFC 4180 + PG superset: doubled-quote escape, embedded-delimiter/quote/newline quoting, empty-unquoted = NULL, empty-quoted = empty-string (distinct), custom NULL marker, record-oriented parser reassembles quoted-newline records across CopyData frame boundaries. HEADER on input drops the first record; on output emits the column names as a leading CopyData. Inherits SP-PG-COPY-BULKAPPLY V1 batching

  • NULL-fallback semantics — CSV is just a different payload codec at the dispatcher. HEADLINE on vulcan: psql 16 COPY FROM CSV HEADER (3 rows including embedded comma + doubled-quote escape) + COPY TO CSV HEADER round-trip byte-equal. Custom DELIMITER ';' + NULL '' verified end-to-end. Unlocks pg_dump --csv, psql \copy ... CSV HEADER, and every spreadsheet/pandas analyst on-ramp. FORCE_QUOTE / FORCE_NOT_NULL / FORCE_NULL → precise 0A000 with V2 arc names (SP-PG-COPY-CSV-FORCEQUOTE); non-UTF-8 ENCODING → 0A000 (SP-PG-COPY-CSV-ENCODING); HEADER MATCH (PG-15+) → V2 SP-PG-COPY-CSV-HEADER-MATCH. Smoke transcript: docs/superpowers/sppgcopycsv-t2-smoke-2026-06-01.txt. KAT delta: +24 (copy::csv::* + copy::dispatch::csv_* + copy::command::csv_*). Arc closed — TaskList #358 ready for completion.

• Track A.2.2 — PostgreSQL COPY binary format (SP-PG-COPY-BIN V1 SHIPPED — 2026-06-02). WITH (FORMAT binary) accepted for both COPY FROM STDIN and COPY TO STDOUT. Per PG §55.2.7: 19-byte signature header (PGCOPY\n\xff\r\n\0 + 4-byte flags + 4-byte header extension length), per-row 2-byte BE i16 field count + per-field 4-byte BE i32 length (-1 = NULL) + binary-encoded value, 2-byte BE i16 -1 end-of-data marker. Same 10 supported types as SP-PG-EXTQ-BIN-RESULTS (BOOL, INT2/INT4/INT8, FLOAT4/FLOAT8, TEXT/VARCHAR, BYTEA, TIMESTAMPTZ) via direct reuse of extq::binary_results::encode_binary_value (TO) and extq::substitute::decode_binary_param (FROM). NUMERIC since closed through SP-PG-COPY-BIN-NUMERIC V1 (2026-06-02 — Track A.-1.5). Tables with UUID / JSONB / ARRAY columns continue to pre-reject at COPY-start with precise V2-arc-pointing 0A000 messages (SP-PG-COPY-BIN-EXTRA); session stays alive. Inherits SP-PG-COPY-BULKAPPLY V1 batching throughput (binary values are decoded back to text before the existing per-row INSERT synthesizer — trade-off named in design §9.1 as the V2 SP-PG-COPY-BIN-DIRECT lift). HEADLINE on vulcan: psql 16.14 CREATE TABLE + INSERT seed + COPY t TO STDOUT WITH (FORMAT binary) to file + COPY t2 FROM STDIN WITH (FORMAT binary) into fresh table + SELECT * → same row set + re-export byte-equal (md5sum match d4df79da...). Unlocks pg_dump --format=custom restore, JDBC CopyManager.copyIn(PGCopyOutputStream...), pg_bulkload, pgloader, Stitch, Fivetran, Airbyte binary bulk-loaders. Smoke transcript: docs/superpowers/sppgcopybin-t3-smoke-2026-06-02.txt. KAT delta: +31 (copy::binary::* + copy::proto::binv1_* + copy::command::t1_parse_copy_binary_format_accepted_in_v1

  • server::tests::t2_run_session_copy_binary_format_accepted_v1). Arc closed — TaskList #360 ready for completion.

• Track A.3 — PostgreSQL COPY throughput (SP-PG-COPY-BULKAPPLY V1 SHIPPED — 2026-05-30). COPY FROM STDIN now buffers up to COPY_BATCH_SIZE rows (default 1024, env-overridable via KESSELDB_COPY_BATCH_SIZE) and flushes each batch as ONE multi-row INSERT INTO t (cols) VALUES (...), (...), ..., which kessel-sql compiles to Op::Txn { ops: Vec<Op::Create> } — one apply round-trip + one WAL fsync per batch instead of one per row. HEADLINE — 100K-row COPY on vulcan: 1.929s = 51,840 rows/sec (median of 3 trials), a 181.9× lift over the V1 baseline 285 rows/sec. KesselDB now within ~11× of Postgres 16 (578,034 rows/sec) on the same workload (was ~2000× behind). Per-batch atomicity: each batch is an Op::Txn and rolls back whole on any inner failure (documented divergence vs PG's whole-COPY atomicity — SP-PG-COPY-BULKAPPLY-WHOLECOPY named as follow-up arc, gated on engine-side streaming-Txn shape). NULL-row fallback preserves correctness for nullable schemas (each NULL-containing batch falls back to per-row dispatch; all-non-NULL batches get the headline lift). Bench transcript: docs/superpowers/sppgcopybulkapply-t3-bench-2026-05-30.txt. Named follow-up arcs: SP-PG-COPY-BULKAPPLY-WHOLECOPY (full PG- compatible atomicity), SP-PG-COPY-BULKAPPLY-NULLBATCH (restore the BULKAPPLY win for NULL-heavy batches). Arc closed — TaskList #351 ready for completion.

• Track B — Perf-A read-pool arc (T1 → T7) + TXN-RO follow-on. Parallel-read bypass (read_only_op(&self, ...) dispatch through Arc<RwLock<StateMachine>>) + storage Arc<[u8]> migration on the read fast path: 4.75M ops/sec at N=16 cores, p50 < 1 µs, p99 ~3 µs. Storage point-read ceiling honestly diagnosed at ~5M ops/sec (RwLock reader CAS ping-pong). Follow-on SP-Perf-A-TXN-RO V1 SHIPPED (2026-05-29) — all-RO Op::Txn{ops} now classified statically + routed through the same bypass, closing the sysbench oltp-read-only loss (N=16 680 → 28,977 tx/s, 42.6× lift, now 5.7× faster than Postgres). Next arcs named: SP-Perf-A-TXN-RW (mixed-RW Op::Txn via SI + commit-time conflict detection) + SP-Perf-A-SHARD (sharded apply queues + per-shard read pools).

• Track C — Cross-DB benchmark suite (SP-Bench-Suite T1-T5). YCSB-A/B/C (KesselDB wins) + sysbench OLTP RO/WO/RW (KesselDB wins WO decisively, loses RO/RW to Postgres+SQLite — root cause: Op::Txn apply-lock held for the whole bracket even when every inner op is read-only) + TPC-H Q1/Q6 (pre-arc KesselDB lost both — Postgres uses shipdate index narrowing, KesselDB did full-scan + per-row VM eval; SP-Analytic-Plan (2026-05-29) closed the Q6 gap 7.5×, 123×→16× vs Postgres). Two roadmap arcs named: SP-Perf-A-TXN-RW (closes sysbench RW; RO already CLOSED by SP-Perf-A-TXN-RO 2026-05-29) + SP-Analytic-Plan-MULTI (the second prong for Q1 — folds 4 scans into 1 via Op::GroupAggregateMulti; T4 first prong already lifted Q1 1.15× via range_preds). Wins AND losses published verbatim in docs/BENCHMARKS.md. Arc closed at T5; T6 final-sweep remains.

• Track E — SP-Analytic-Plan (2026-05-29, V1 SHIPPED). Closes the SP-Bench-Suite T4 TPC-H Q6 loss by teaching Op::Aggregate + Op::GroupAggregate to consume the range_preds: Vec<(field_id, op, value)> interface already shipping in Op::QueryRows (SP70). T1 design + scaffold (additive proto field, wire-back-compat preserved). T2 kessel-sm apply paths use a shared narrow_by_range_preds helper that intersects candidate row-ids via the existing 0xFFFD/0xFFFC ordered-index keyspaces BEFORE the per-row WHERE program runs (the program still verifies every candidate, so the aggregate result is byte-identical to a full-scan oracle — proven by 3 equivalence KATs across COUNT/SUM/MIN/MAX/AVG and empty/singleton/full-cover windows). T3 kessel-sql compile_select aggregate branch emits range_preds via a shared extract_range_preds helper (same conjunct-safety gate as try_query_rows); proven end-to-end by an indexed-vs-unindexed-twin KAT across 7 SQL shapes. T4 bench-compare TPC-H driver adds Op::AddOrderedIndex on l_shipdate + range_preds on Q1/Q6 ops. Headline on vulcan (3-trial median × 30s × SF=0.01 ≈ 60K rows): Q6 N=1 3.53 → 25.39 q/s (7.2×), Q6 N=4 13.74 → 103.38 q/s (7.5×) — gap vs Postgres closed from 123× to 16×; Q1 N=1 2.38 → 2.80 q/s (1.18×), Q1 N=4 8.84 → 10.14 q/s (1.15×) — small because Q1's WHERE covers ~all rows (the multi-aggregate fold is the next prong, SP-Analytic-Plan-MULTI). Workspace tests: 2018 → 2024 default (+6 new KATs: 1 proto wire-back-compat, 3 SM equivalence, 2 SQL planner integration). seed-7 GREEN; CI green at HEAD 8726157; #![forbid(unsafe_code)] honored; zero new external deps; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched.

• Track F — SP-Perf-A-TXN-RO (2026-05-29, V1 SHIPPED). Closes the SP-Bench-Suite T3 sysbench OLTP read-only loss (KesselDB was LOSING at every N≥8 because Op::Txn{ops} was routed through StateMachine::apply() even when every inner op was a read — the Perf-A T2 read-pool bypass was GetById-only and didn't compose with Op::Txn). Five slices T1-T5 all DONE: T1 design spec + progress tracker; T2 server-side classifier (read_pool::is_read_only) now recurses into Op::Txn { ops } and returns true iff every inner op is read-only; T3 StateMachine::read_only_op gains an Op::Txn arm that mirrors apply-Txn's 15-variant data-op contract EXACTLY (SeqRead permitted bare-Op but rejected inside Txn; verbatim error string match for divergence-via-string-eq safety) plus dispatch wiring (apply_raw tag-15 + in-process apply classifier swap) plus determinism oracle extension (txn_ro_oracle_100_workloads_x_1000_txns_byte_equal

  • 7 per-shape smoke KATs covering empty Txn, single inner, sysbench shape (410 inner ops), 15 permitted variants, SeqRead-rejection symmetry, mixed-RW falls through, write-at-front falls through); T4 bench-compare driver routes RO Txns via sm.read().unwrap().read_only_op(Op::Txn{ops}); T5 STATUS + arc closure. HEADLINE on vulcan (3-trial median × 10s × 10×100K rows): oltp-read-only N=1 1,241 → 2,299 tx/s (1.85×); N=8 641 → 16,213 tx/s (25.3×); N=16 680 → 28,977 tx/s (42.6×) — gate was ≥3000 at N=16; beaten 9.7×. KesselDB now BEATS Postgres by 4.0× at N=8 and 5.7× at N=16 (was LOSING by 6.3× / 7.5×). p50 at N=8 dropped from 12.6 ms to 475 µs (26× faster). oltp-RW unchanged within noise as designed (mixed-RW V1 limit; named follow-up SP-Perf-A-TXN-RW). Workspace tests: kesseldb-server lib 137 GREEN (+22 new test-binary tests); seed-7 GREEN; #![forbid(unsafe_code)] honored; zero new external deps; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched; default cargo build -p kesseldb-server byte-identical (the classifier extension + SM arm are additive; is_mutating() in proto unchanged so VSR / replication / op-number assignment all carry on as before). Five commits: fc8baff (T1 design), e2479ec (T2 classifier), 3dbe8fe (T3 SM arm + dispatch + oracle), 75001e5 (T3 SeqRead-rejection-mirror fix), fcff211 (T3 per-variant bisect), 4ebb338 (T3 smoke 4 GetBlob{0} fix), plus this commit (T4 bench sweep + T5 closure). Progress tracker docs/superpowers/specs/2026-05-29-kesseldb-spperfa-txnro-progress.md CLOSED. Arc closed — TaskList #341 ready for completion.

• Track G — SP-Analytic-Plan-MULTI (2026-05-30, V1 SHIPPED). Closes the SP-Analytic-Plan T4 residual TPC-H Q1 gap (was 18× behind Postgres). New Op::GroupAggregateMulti { aggregates: Vec<(kind, field_id)>, range_preds, … } at wire tag 47 — additive new variant; existing Op::Aggregate (20) + Op::GroupAggregate (22) wire bytes byte-identical (back-compat). Folds N aggregates (COUNT/SUM/MIN/ MAX/AVG) per row in ONE scan instead of N×Op::GroupAggregate calls, collapsing the per-row WHERE-eval + group-key-extract cost from N× to 1×. T1 design + scaffold + wire KAT (3 vectors covering Q1 shape). T2 SM apply paths via shared group_aggregate_multi() helper used by BOTH apply + read_only_op (byte-identical results guaranteed) + 3 equivalence KATs (vs N×Op::GroupAggregate, apply vs read_only_op, full-cover range_preds invariant). T3 kessel-sql compile_select projection parser refactored to accept comma- separated mix of leading group cols + aggregate calls; emits Op::GroupAggregateMulti for ≥2 aggregates / leading-col + ≥1 agg (single-agg paths byte-identical, plain-col-after-agg + multi-agg- without-GROUP-BY rejected). T4 bench-compare TPC-H Q1 driver uses one Op::GroupAggregateMulti carrying 4 aggregates instead of 4 separate Op::GroupAggregate + client-side BTreeMap merge. HEADLINE on vulcan (3-trial median × 30s × SF=0.01 ≈ 60K rows): Q1 N=1 2.80 → 10.90 q/s (3.89×), Q1 N=4 10.14 → 41.11 q/s (4.05×) — gap vs Postgres closed from 18× to 4.5×; KesselDB N=4 now BEATS SQLite N=4 (41.11 vs 23.75 = 1.73× win, was 2.3× loss). The design predicted 3-4× lift band — measured 3.9-4.0× lift is exactly on prediction. The remaining 4.5× Q1 gap is parallel hash aggregate (next arc, SP-Hash-Agg). Workspace tests: kessel-proto 15 → 16, kessel-sm 151 → 154, kessel-sql 38 → 40, kesseldb-server read_pool 33 GREEN (variant count 46 → 47). seed-7 GREEN (partition_corpus_is_deterministic); zero new external deps; #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched. Six commits: d0aa4e4 (T1 design), eb1a417 (T1+T2 scaffold + SM helper), c74e74a (T2 equivalence KATs), 60345a3 (T3 SQL planner + KATs), d48d3c4 (T4 bench driver), ff35ed9 (T4 read_pool variant fix), plus this commit (T5 closure). Progress tracker docs/superpowers/specs/2026-05-30-kesseldb-spanalyticplanmulti-progress.md CLOSED. Arc closed — TaskList #342 ready for completion.

• Track J — SP-Hash-Agg (2026-05-30, V1 SHIPPED — DONE_WITH_CONCERNS). Closes the SP-Analytic-Plan-MULTI residual TPC-H Q1 + Q6 gaps vs Postgres' parallel hash aggregate by parallelising the per-row aggregate-fold across N=4 worker threads within a single query. std::thread::scope + per-worker HashMap partials + sorted-BTreeMap merge for ascending-key output. Zero new external deps (std-only since Rust 1.63); #![forbid(unsafe_code)] honored. Two-phase materialise + parallel-fold: Phase A (dispatcher) collects candidate rows into Vec<Arc<[u8]>> (Arc keeps the storage.get refcount path zero-memcpy per SP-Perf-A T7; scan_range results wrapped in Arc to unify the per-worker chunk type); Phase B (4 workers) each fold one row-offset chunk into a local HashMap partial (or scalar accumulator for Op::Aggregate); Phase C merges partials in deterministic (0..N) order into a sorted BTreeMap. Combine ops are associative for SUM/ COUNT and associative+commutative for MIN/MAX; AVG computed POST-merge from (sum, count) via integer division (matches serial path byte-for- byte). MIN_PARALLEL_ROWS = 8192 gates the parallel path; below threshold the existing single-threaded fold runs verbatim (zero overhead for OLTP-shape aggregates). T1 design + scaffold + constants. T2 SM apply paths: aggregate_numeric_scan helper added (replaces ~280 lines of inline-duplicated loop) called from both Op::Aggregate apply arms; group_aggregate_multi rewritten with the parallel path. T3 three new SM-level equivalence KATs lock parallel == serial byte-for-byte at scale (10K rows × Q1-shape Multi, 10K rows × Q6-shape Aggregate, apply == read_only_op at scale). T4 vulcan TPC-H Q1+Q6 sweep (3 trials × 30s × SF=0.01 × N=1,4 × 3 per-cell trials = 9 trials/cell). HEADLINE on vulcan: Q1 N=1 10.90 → 17.30 q/s (+1.59×), Q1 N=4 41.11 → 60.18 q/s (+1.46×); Q6 N=1 25.39 → 34.23 q/s (+1.35×), Q6 N=4 103.38 → 185.03 q/s (+1.79×). Cumulative 3-arc lift vs pre-arc baseline (SP-Bench-Suite T4): Q1 N=4 +6.81×; Q6 N=4 +13.47×. Gap-closing vs Postgres: Q1 N=4 4.52× → 3.09× (was 18× pre-arc); Q6 N=4 16× → 9.11× (was 123× pre-arc). DONE_WITH_CONCERNS: design predicted 4× per-query lift (4-way row-chunk parallelism), measured 1.5×. Diagnosis (BENCHMARKS.md §3f honest read): the serial prefix (Vec<Arc<[u8]>> materialisation of the candidate row set + thread- spawn cost at 4 workers) is hard-pinned to one CPU and accounts for the bulk of wall-time. Named follow-up arcs SP-Hash-Agg-Tune (streaming materialisation, thread-pool reuse, bypass Arc::from on the scan_range path; expected 2-3× more) and SP-JIT-Aggregate (LLVM codegen for the per-row inner loop, what Postgres uses). Workspace tests: kessel-sm 154 → 157 (+3); all 15 pre-existing aggregate KATs stay green. seed-7 GREEN; zero new external deps; #![forbid(unsafe_code)] honored; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched. Five commits: 49d318c (T1 design + progress tracker + MIN_PARALLEL_ROWS const), fa30246 (T2 parallel hash aggregate for Op::Aggregate + Op::GroupAggregateMulti), 21d0b8b (T3 equivalence + determinism KATs), 5b0fb14 (T4 BENCHMARKS.md §3f/§3g/§1 update), plus this commit (T5 STATUS + progress tracker close + README). Progress tracker docs/superpowers/specs/2026-05-30-kesseldb-sphashagg-progress.md → DONE_WITH_CONCERNS. TaskList #345 ready for completion.

• Track K — SP-Hash-Agg-Tune (2026-05-30, V1 SHIPPED — DONE_WITH_CONCERNS). Drives down the SP-Hash-Agg V1 serial-prefix cost. V1 used a pre-collect Vec<Arc<[u8]>> + chunk-then-spawn shape that paid the FULL row materialisation cost SERIALLY before any worker spawned (1.46-1.79× lift measured vs 4× modelled — V1 progress tracker named SP-Hash-Agg-Tune as the residual-cost arc). V1-Tune rewrites both aggregate_numeric_scan (Q6) + group_aggregate_multi (Q1) with producer-channel-workers BATCHED streaming: one producer thread iterates the source (Pre or Scan), packs rows into BATCH_SIZE=256 Vec batches, sends round-robin into N=4 bounded sync_channel(BUF_DEPTH=16); N=4 worker threads each consume their channel batch-at-a-time and fold rows AS THE BATCH ARRIVES. Workers start on row 1 instead of row LAST, overlapping producer iteration with worker fold. T1 design + scaffold + streaming refactor (unbatched first — commit 833eede); intermediate shape regressed -13%/-9% at N=1/N=4 because per-row channel send/recv (60K rows × ~500ns = ~30ms/query) SWALLOWED the streaming savings; T2.1 batched fix (0a19f3d) amortises channel cost across BATCH_SIZE=256 rows. T2 streaming-equivalence KATs (3 new sp_hash_agg_tune_*): 9K-row BUF_DEPTH stress + 50K-row × 100-group high-cardinality + 15K-row apply==read_only_op at scale. T3 vulcan TPC-H Q1+Q6 sweep (3 trials × 30s × SF=0.01 × N=1,4). HEADLINE on vulcan (post-Tune BATCHED): Q1 N=1 17.30 → 16.14 q/s (-1.07×), Q1 N=4 60.18 → 63.77 q/s (+1.06×); Q6 N=1 34.23 → 33.95 q/s (par), Q6 N=4 185.03 → 197.55 q/s (+1.07×). Cumulative 4-arc lift vs pre-arc baseline (SP-Bench-Suite T4): Q1 N=4 +7.21×; Q6 N=4 +14.38×. Gap-closing vs Postgres: Q1 N=4 3.09× → 2.92×; Q6 N=4 9.11× → 8.53×. DONE_WITH_CONCERNS: user-spec floors (Q1 ≥120 / Q6 ≥350 q/s at N=4) MISSED — 53% / 56% achieved. New diagnosis from the sweep: the V1 serial Arc-wrap pre-collect was NOT the dominant wall-time cost (V1-Tune eliminated it via streaming overlap, gained only +6-7%). The actual dominant cost is the per-row kessel_expr::eval stack VM interpreter evaluating the WHERE program ~60K (Q1) / 8K (Q6) times per query — the row-chunk parallel fold can amortise it across cores but cannot make per-row eval cheaper. Named follow-up arcs SP-WHERE-VM-Specialise (closure-built-once-per-query that inlines field offsets + comparison ops; expected 1.5-2× per row) and SP-JIT-Aggregate (LLVM/cranelift codegen for the per-row inner loop; what Postgres uses; closes the constant-factor gap). SP-Hash-Agg-Pool de-prioritised (V1-Tune sweep showed thread-spawn is NOT the bottleneck). Workspace tests: kessel-sm 157 → 160 (+3 new KATs); all 6 SP-Hash-Agg + SP-Hash-Agg-Tune KATs green. seed-7 GREEN; zero new external deps; #![forbid(unsafe_code)] honored (sync_channel

  • thread::scope are safe std); HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched. Three commits: 833eede (T1+T2 design + streaming refactor + KATs), 0a19f3d (T2.1 BATCHED channel sends), plus this commit (T3 BENCHMARKS.md + T4 STATUS + tracker close + README). Progress tracker docs/superpowers/specs/2026-05-30-kesseldb-sphashaggtune-progress.md → DONE_WITH_CONCERNS. TaskList #347 ready for completion.

• Track L — SP-Perf-A-SHARD-1 (2026-05-30, design + scaffold + K=1 regression-lock LANDED; multi-arc continuation NAMED). Attacks the SP-Perf-A T7 ~5M ops/sec ceiling diagnosed as RwLock<StateMachine> reader-count CAS ping-pong between cores. SHARD partitions the key space into K per-CPU shards, each its own Arc<RwLock<StateMachine>>

  • read lock; readers on shard 0 don't contend with readers on shard
  1. Honestly scoped as multi-arc: SHARD-1 (this slice) ships design + scaffold + K=1 regression-lock; the K=N apply plumbing is multi-week core work named SP-Perf-A-SHARD-APPLY (V2). T1 design spec (11 sections + 8 weak-spots + 7 locked invariants + 6-arc decomposition: SHARD-APPLY, SHARD-READ, SHARD-SCAN, SHARD-XTXN, SHARD-BENCH). T2 scaffold: crates/kesseldb-server/src/sharded_sm.rs with ShardedStateMachine<V>, shard_of_key (K=1 short-circuit + K>=2 fxhash-mod), shard_of_op (point ops → Single, scans / joins / cross-shard Txn → FanOut), read_only_op_k1 (panics on K>=2 as fail-fast against stale K=N configs). 11 KATs (all green on vulcan) including the headline shard_k1_matches_unsharded_sm_byte_equal regression-lock — seeds two state machines identically, wraps one in a K=1 ShardedStateMachine, asserts byte-equal read_only_op results across hit/miss/Describe ops. ServerConfig.shard_count: Option<usize> field added but NOT wired into spawn_engine_cfg (engine wiring is SHARD-APPLY's job); default None preserves SP-Perf-A T7 ownership shape. No throughput lift in this slice — named scope was design + scaffold, NOT measurement. That's SHARD-BENCH's job once SHARD-APPLY + SHARD-READ + SHARD-SCAN merge. Workspace tests: kesseldb-server lib 148 → 159 (+11 SHARD tests, 0 regressions); kessel-sim release 3/3 green; cargo build --workspace clean; #![forbid(unsafe_code)] honored; zero new external runtime deps (fxhash_fold inline, 8 lines). Two commits: f634f07 (T1 design + tracker), d5691a6 (T2 scaffold + 11 KATs), plus this commit (tracker T2 done + STATUS row + README untouched). Progress tracker docs/superpowers/specs/2026-05-30-kesseldb-spperfa-shard-progress.md → PAUSED at SHARD-1 DONE (multi-arc continuation named). TaskList #348 partial progress — design + scaffold landed; K=N apply path is the SP-Perf-A-SHARD-APPLY sub-arc.

• Track L cont. — SP-Perf-A-SHARD-APPLY (2026-05-30, K=N apply path SHIPPED; vulcan 3.19× lift at K=8 BREAKS the 10M ops/sec ceiling). The multi-week-core arc named in SHARD-1 — wires K independent per-shard sub-engines (each its own Arc<RwLock<StateMachine>> + apply thread + WAL + SSTables, rooted at data_dir/shard-<i>/) and routes every Op via hash(make_key(type_id, oid)) % K. T1: crates/kesseldb-server/src/sharded_engine.rs with ShardedDispatcher, route_op classifier (Single(s) for point ops by primary-key shard; per-type pinning for FindBy / Describe / FindRange / FindByComposite via (type_id, zero-oid); sequencer pinned via fixed SEQ_TYPE key; Broadcast for every DDL op including CreateType / CreateIndex / AddOrderedIndex / AddCompositeIndex / AddUnique / AddForeignKey / AddCheck / AddTrigger / AddBalanceGuard / Drop* / RenameField / AlterTypeAddField / Create|Drop|Refresh-ExternalSource; ShardZero for scans / Txn / cross-shard ops as documented V1 limitation). spawn_sharded_engine_cfg spawns K vanilla sub-engines via spawn_engine_cfg(.., shard_count=None)

  • a router-shell engine at data_dir/router/ whose EngineHandle.sharded = Some(dispatcher); apply_raw / apply / apply_op short-circuit through the dispatcher when set. Activation: opt-in via ServerConfig.shard_count = Some(K) with K >= 2; default None and Some(1) preserve SP-Perf-A T7 ownership shape byte-for-byte (SHARD-1 K=1 regression-lock KAT still green). T2: 4 integration KATs incl. headline t2_determinism_oracle_k1_k4_k8_byte_equal (seeds identical 100-row workload on K=1 / K=4 / K=8 engines, asserts byte-equal GetById + Describe results across all K). T3: --shard-count N flag on kessel-bench parallel-reads so the same harness measures K=1 / 2 / 4 / 8 / 16. T5 (vulcan YCSB-C sweep, 16 workers, 10K rows, 10s): K=baseline 4.68M ops/sec; K=2 7.30M (1.56×); K=4 11.08M (2.37× — blows past 6M target); K=8 14.93M (3.19× — BREAKS the 10M ceiling, the HEADLINE TARGET); K=16 16.72M (3.57× — diminishing return curve starting to flatten, V2 SHARD-READ would push further). p50 latency drops from 3 µs (unsharded) to <1 µs (K>=4). Test surface: kesseldb-server lib 159 → 172 tests (+13 SHARD-APPLY: 9 routing classifiers + 4 end-to-end KATs); 172/172 green; default cargo build byte-identical; #![forbid(unsafe_code)] honored; zero new external runtime deps. Honest V1 limitations: scan ops (Select / Aggregate / Query / Join / etc.) route to shard 0 ONLY — INCORRECT for data spread across shards (named SP-Perf-A-SHARD-SCAN follow-up); Op::Txn routes to shard 0 (cross-shard Txn = SP-Perf-A-SHARD-XTXN follow-up); VSR × sharding is its own arc. Commits: 76d5a50 (T1 per-shard engine + routing), 37371fd (T2 oracle KATs), 27e3092 (T3 bench flag), plus this commit (T5 benchmark results + T6 STATUS + BENCHMARKS §13 + tracker close). Progress tracker → SHARD-APPLY DONE (continuation arcs SHARD-READ / SHARD-SCAN / SHARD-XTXN / SHARD-BENCH-full remain named). TaskList #349 DONE — K=N apply plumbing is the multi-week core SHARD-1 named; today's slice ships it AND lifts the ~5M ops/sec ceiling to 14.93M.

• Track L cont. — SP-Perf-A-SHARD-SCAN (2026-05-30, scatter-merge for scan ops at K>=2 SHIPPED — production-correctness fix). SHARD-APPLY left a known gap: scan ops (Select / QueryRows / SelectFields / SelectSorted / Aggregate / GroupAggregate / etc.) routed to shard 0 ONLY at K>=2, returning ~1/K of the data. This arc wires the SP-A scatter-merge machinery (scatter_scan.rs, already in production use by the cluster router for network-attached shards) into the in-process sharded engine via a new InProcShardCaller impl of ShardCaller (calls EngineHandle::apply_op directly — zero network, zero serialization). Same machinery, same merge contract, different transport. Routing reclassification: 12 scan ops (Select / QueryRows / SelectFields / SelectSorted / Aggregate / GroupAggregate / GroupAggregateMulti / FindBy / FindByComposite / FindRange / Query / QueryExpr) all switch from ShardZero to Scatter(ScatterKind). Three NEW ScatterKind variants added: OidSortedUnion (sort+dedup oid union for Query/QueryExpr/FindRange whose K=1 baseline sort_unstable+dedups), AggregateMerge { kind, field_kind } (COUNT/SUM sum i128s; MIN/MAX pick numeric ≤8B vs var-width path), GroupAggregateMerge { kind } / GroupAggregateMultiMerge { kinds } (BTreeMap-based per-group combine). Catalog-dependent params (Sorted's sort-field byte offset

  • width; AggregateMerge's MIN/MAX field_kind) resolved at dispatch time via Op::Describe against shard 0 — mirrors cluster router's scatter_read pattern. T1+T2: 14 new KATs (12 merge function + 2 routing classification). T3: K-invariance oracle — 100-row workload × 12 scan ops × K∈{1,4,8} asserts byte-equal (Sorted/Aggregate/GroupAggregate/OidSortedUnion) or multiset-equal (Unordered/OidConcat) (t3_shard_scan_k_invariance_oracle_12_ops green; supplemented by t3_shard_scan_group_agg_byte_equal_uneven_groups for non-uniform group sizes and t3_shard_scan_aggregate_avg_asymmetric_k1_vs_kn documenting the AVG limitation). T4: vulcan bench sweep across select-limit / select-sorted / aggregate-sum / find-by × K∈{1,4,8} — results in BENCHMARKS §14. Honest V1 limitations: (1) Op::Aggregate kind=4 (AVG) hard-fails at K>=2 because per-shard reply is sum/count without per-shard count — SHARD-SCAN-AVG follow-up changes the wire shape; K=1 AVG unchanged. (2) Op::Join unchanged (cross-shard join is SHARD-JOIN's job). (3) SHARD-APPLY's per-type pin still exists (redundant for correctness now but kept to avoid invalidating on-disk shard layouts; SHARD-APPLY-2 lifts it). (4) Cross-shard scan snapshot consistency requires MVCC seq plumbing (SHARD-SCAN-SNAPSHOT). Test surface: kesseldb-server lib 172 → 188 tests (+16; 0 regressions); workspace clean; #![forbid(unsafe_code)] honored; zero new external runtime deps; default cargo build byte-identical (new routing classifications only activate when shard_count >= 2). Vulcan bench sweep (T4, --pool-workers 16, 10K rows, 10s): select-limit K=4 = 0.75× / K=8 = 0.64× (LIMIT 10 = per-shard does ~4×/8× excess scan work then merges to 10 — measured regression); select-sorted K=4/8 ≈ 1.0× (k-way heap merge overhead ≈ per-shard scan savings); aggregate-sum K=4 = 1.18× lift (full-scan SUM fans out, K=4 is the sweet spot; K=8 = 0.87× as routing overhead dominates); find-by K=4 = 0.006× (1.8M → 10K ops/sec — secondary-index lookup is sub-microsecond at K=1, thread-spawn overhead of scatter-merge ~1500µs vs ~500ns direct path causes massive structural regression on point-shaped indexed lookups). Honest verdict: SHARD-SCAN ships the correctness fix (12 scan ops now return right answers at K>=2 instead of 1/K). Perf is workload-dependent: large-scan aggregates benefit at K=4; small-result-set indexed lookups regress significantly. Named follow-up SHARD-SCAN-FASTPATH would short-circuit tiny-result-set ops to avoid per-request thread-spawn — could recover 100×+ of the find-by overhead. Commits: 1d2fcb1 (T1+T2 scaffold + routing + 14 KATs), 72287fe (T3 K-invariance oracle + 3 KATs), plus this commit (T4 bench + T5 STATUS + BENCHMARKS §14 + tracker close). Progress tracker → SHARD-SCAN V1 SHIPPED — DONE for correctness; DONE_WITH_CONCERNS for perf shape (named SHARD-SCAN-FASTPATH follow-up). TaskList #352 ready.

• Track L cont. — SP-Perf-A-SHARD-SCAN-POOL-SCALEOUT (2026-06-01, V1 SHIPPED). Closes the select-limit / select-sorted / aggregate- sum regressions FASTPATH (2026-05-30) left open. Approach A (T1 — bump sync_channel(1) to sync_channel(64)) was tested on vulcan and proved insufficient: K=4 numbers for select-limit / select- sorted / aggregate-sum were UNCHANGED from POST-FASTPATH (949 vs 958; 214 vs 214; 941 vs 937), because per-worker throughput, not channel backpressure, was the bottleneck — 16 dispatchers always serialize through K=4 workers no matter how big the per-worker queue is. T2/T4 escalated to Approach C from the design spec: refactor ScatterPool to spawn M = max(K * 4, 16) workers sharing a single mpsc::sync_channel(POOL_BOUND) queue, with per-shard dispatch closures held in Arc<Vec<Box<dyn Fn>>> shared by every worker. Work items carry shard_id: u32; any worker can fulfill any (shard_id, op) pair. Vulcan bench (single trial, 10K rows, 16 workers, 10s): select-limit K=4 = 3,169 ops/sec (3.31× lift from POST-FASTPATH 958, 1.23× FASTER than K=1 baseline 2,571); select-sorted K=4 = 802 (3.75× lift from 214, 1.19× faster than K=1 674); aggregate-sum K=4 = 3,044 (3.25× lift from 937, 2.06× faster than K=1 1,478); find-by K=4 = 1,057,854 (preserved within 0.8% of FASTPATH's 1,066K headline). K=8 numbers similarly lift: select-limit 4,175 (2.28×), select-sorted 877 (1.98×), aggregate-sum 3,170 (1.67×), find-by 836K (preserved). Every scan workload at K=4 now scales POSITIVELY with K — what FASTPATH framed as "corner-case regressions" is no longer regressed. K-invariance oracle still GREEN (12 scan ops byte/multiset-equal across K∈{1,4,8}). Test surface: kesseldb-server lib 198 → 202 (+4; +1 KAT for POOL_BOUND constant, +1 KAT for 16-dispatcher-deadlock sanity, +1 KAT for M worker-count formula, +1 KAT for shard_id routing under shared workers). Default cargo build byte-identical; #![forbid(unsafe_code)] honored; zero new external deps (std::sync::Mutex only). Commits: 0d9f221 (T1 — POOL_BOUND 1 → 64 + KAT, proved insufficient), 850c43d (T2/T4 — Approach C escalation + Arc<Vec> refactor + shared-queue worker loop + 2 KATs), plus this commit (T3 bench + BENCHMARKS §14c + tracker close). Progress tracker → SHARD-SCAN-POOL-SCALEOUT V1 SHIPPED. TaskList #354 ready.

• Track L cont. — SP-Perf-A-SHARD-SCAN-FASTPATH (2026-05-30, V1 SHIPPED). Closes the find-by perf regression SHARD-SCAN named. Two complementary fixes: (A) persistent ScatterPool — K long-lived worker threads block on sync_channel(1) waiting for work; replaces per-call std::thread::spawn (per-call overhead drops from ~1500µs to ~10-100µs); (B) serial fast path for tiny scans — for Op::FindBy / Op::FindByComposite (sub-microsecond indexed lookups), walk every shard sequentially on the dispatcher thread (no channel hop, no pool dispatch). is_tiny_scan(op) predicate classifies at routing time; scatter_serial does the walk + the same merge_scan_results call as the parallel path. Vulcan bench (3-trial median): find-by K=4 = 1,066K ops/sec (105× lift from 10K, recovers to 59% of K=1 baseline 1,810K); K=8 = 844K (185× lift from 4.5K, 47% of K=1). Both crush the spec's 50× / 25× recovery targets and the 2× K=1 target. Other workloads mixed: aggregate-sum K=8 = 1,897 (1.30× over K=1); select-limit/select-sorted at K=4 regressed further due to pool channel contention (16 dispatcher threads → 4 workers under saturation) — named follow-up SHARD-SCAN-POOL-SCALEOUT (per- dispatcher pool replicas). K-invariance oracle still GREEN; 12 scan ops still byte/multiset-equal across K∈{1,4,8}. Test surface: kesseldb-server lib 188 → 198 (+10; 8 ScatterPool KATs + 2 Approach-B KATs). Default cargo build byte-identical (pool only constructed when shard_count >= 2). #![forbid(unsafe_code)] honored; zero new external deps. Commits: 01cbbb6 (T1+T2 design + ScatterPool scaffold + dispatcher wire-up + 8 KATs), af98f3a (Approach B serial fast path + 2 KATs), plus this commit (T3 bench + T4 STATUS + BENCHMARKS §14b + tracker close). Progress tracker → SHARD-SCAN-FASTPATH V1 SHIPPED. TaskList #353 ready.

• Track D — Cluster test flakes (SP-CLUSTER-FLAKE T2). Root-cause fixed in Node::submit* / apply_raw: production VSR retry on transient ViewChange. Not just a test relaxation — the actual production code path now retries Unavailable the same way ClusterClient does. CI green at HEAD 546e79a.

• Track H — SP-DX-superior (2026-05-30, V1 SHIPPED). Developer-experience audit on top of the perf + protocol wins. Three concrete shipments, each individually load-bearing for first-5-minutes adoption:

  1. Better errors (T1). unknown table now suggests the closest match in the live catalog via a zero-dep edit-distance + prefix matcher; on an empty catalog the message says "no tables defined yet — use CREATE TABLE first" instead of a bare unknown table \foo`. unknown columnnow includes the owning table name + either a did-you-mean (e.g.owne→owner) or the head of the actual column list, so users never need a separate DESCRIBEround-trip. ThekesselCLI differentiates connection-refused / wrong-token / DNS-failure / timeout — each branch points at the env var or flag that controls that surface. Text + JSON paths strip the duplicative server-sidesql:prefix from SchemaError so users see the friendly inner message directly. (+3 KATs:suggestshape,unknown_tabledid- you-mean,unknown_column` table-context.)
  2. Docker image (T2). Dockerfile at the repo root composes the existing --features pg-gateway,http-gateway release binary into a debian-slim runtime image (77 MiB stripped, ~25 MiB build context via .dockerignore). Image runs as a dedicated non-root kessel:1100 UID; default ENTRYPOINT exposes all three wire surfaces (binary 6532, HTTP+WS 6533, PG 5432). release.yml gains a parallel docker job that builds multi-arch (linux/amd64 + linux/arm64) and pushes to ghcr.io/<owner>/<repo> on every v* tag, tagged :<version>, v<version>, AND :latest for non-prerelease tags. Best-effort (continue-on-error: true) so a registry/QEMU blip can't gate the binary release. Verified end-to-end on vulcan: image builds clean (rust:1-slim base, no system deps), starts cleanly, HTTP gateway accepts CREATE TABLE + SELECT COUNT(*) round-trip.
  3. Embedded example (T3). crates/kesseldb-server/examples/ embedded.rs walks the public in-process API end-to-end: spawn engine with Perf-A read-bypass on, SQL DDL + DML via the new EngineHandle::sql inherent (apply_raw([0xFE]++sql) with a named entry point), typed Op::Create via the codec, hot snapshot. Only depends on already-pinned workspace crates — zero new external dep, zero new feature flag. Verified on vulcan: cargo run --release --example embedded -p kesseldb-server completes in <1 s with all assertions green (SUM(bal) = 1049, kv → [Uint(7), Uint(42)], 3-file snapshot).

  Workspace tests +3 (KATs in kessel-sql for the new error helpers). seed-7 GREEN; #![forbid(unsafe_code)] honored; zero new external deps; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched (the CLI + SQL-compile error rewordings are pure-text changes on the client-side render path; SchemaError variant + wire payload bytes are byte-identical). Default cargo build -p kesseldb-server byte-identical (new EngineHandle::sql is additive). Five commits: c65b010 (T1 errors), e52e9da (T2 Dockerfile + release.yml), 85b8d90 (T2 base-image fix), 33d21c7 (T3 embedded example + EngineHandle::sql), plus this commit (STATUS + USAGE + README). Two follow-ups deferred to focused later slices: SP-DX-INIT (kessel init scaffolder) + SP-DX-REPL (multi-line editor / history in the interactive shell).

• Track I — SP-Perf-A-TXN-RW (2026-05-30, V1 SHIPPED). Closes the SP-Bench-Suite T3 sysbench OLTP read-write loss (KesselDB was LOSING at every N≥8 because mixed-RW Op::Txn{ops} was routed through StateMachine::apply() with the write lock held for the whole 14-op bracket — the SP-Perf-A-TXN-RO bypass was all-RO only and didn't compose with mixed-RW Txn). Five slices T1-T5 all DONE: T1 design spec + progress tracker (with honest architectural pivot from the original full-SI plan — SP112 Tx::write operates at raw MVCC, not at the catalog/index/constraint layer where SM apply's write-arm lives; full SI overlay porting is multi-week and out of V1 scope); T2 server-side classifier read_pool::read_prefix_length(ops) + is_split_safe(suffix) + 11 KATs covering empty/all-R/all-W/ reads-then-writes/(R,W,R)/longer-mixed/canonical-sysbench/etc.; T3 driver-level split-phase dispatch in tools/bench-compare/src/drivers/kesseldb.rs::run_sysbench_oltp — the 3-guard (prefix > 0 && prefix < total && is_split_safe(suffix)) classifies each mixed-RW Txn; eligible Txns split (read prefix via sm.read().read_only_op(Op::Txn{prefix}) parallel + write suffix via sm.write().apply(op_no, Op::Txn{suffix}) serial); ineligible Txns fall through to unified sm.write().apply — plus determinism oracle (1000 random (R[5..15], W[1..4]) Txns unified-vs-split byte- equivalent + sysbench-shape smoke + (R,W,R)-fallthrough smoke); T4 vulcan sysbench OLTP-RW sweep at N=1/8/16 × 3 trials each; T5 BENCHMARKS.md §3e + STATUS + README + arc closure. HEADLINE on vulcan (3-trial median × 10s × 10×100K rows): oltp-read-write N=1 1,472 → 2,088 tx/s (1.42×); N=8 715 → 6,905 tx/s (9.66×); N=16 712 → 10,273 tx/s (14.43×) — gate was ≥3000 at N=16; beaten 3.4×. KesselDB now BEATS Postgres by 2.28× at N=8 and 2.66× at N=16 (was LOSING by 4.22× / 5.43×); also beats SQLite by 1.57× at N=8 and 2.60× at N=16. p50 at N=8 dropped from 11.3 ms to 1.12 ms (10.1× faster). KesselDB scales linearly N=1 → N=16 by 4.92× via the parallel read-prefix dispatch. V1 limit (explicit, documented): read-after-write Txn shapes ((R, W, R) and similar) fall through to unified apply — the 3-guard rejects them for byte-equivalence with apply's overlay-based read-your-writes. For sysbench's canonical (R*, W*) shape this is a no-op. The fallthrough closure is the named V2 follow-up SP-Perf-A-OPTIMISTIC-CC (abort-and-retry with full SI overlay on the SM write path; distinct from the static split- phase shipped here). Workspace tests: kesseldb-server lib 148 GREEN (incl. 11 new read_pool KATs + 3 new parallel_reads_oracle TXN-RW tests); seed-7 GREEN; #![forbid(unsafe_code)] honored; zero new external deps; HTTP/1.1 + WS + binary + PG-wire surfaces byte- untouched (the classifier helpers are pure read-only library functions; the dispatch wiring lives ONLY in tools/bench-compare which is outside the workspace — server bytes unchanged). Default cargo build -p kesseldb-server byte-identical. Three commits: 1fa264b (T1 design + tracker), a93f8a4 (T2 classifier + KATs), fa9b1df (T3 driver dispatch + oracle), 3b854cb (T4 BENCHMARKS update), plus this commit (T5 STATUS + README + tracker close). Progress tracker docs/superpowers/specs/2026-05-30-kesseldb-spperfa-txnrw-progress.md CLOSED. Arc closed — TaskList #344 ready for completion. Both sysbench transaction-bracket losses called out in earlier STATUS revisions are now closed (RO by Track F, RW by Track I). The remaining published losses in the comparison set are the two TPC-H analytical workloads — Q6 already closed 7.5× by SP-Analytic- Plan (Track E), Q1 closed 4× by SP-Analytic-Plan-MULTI (Track G); the residual 4.5× Q1 + 16× Q6 gaps vs Postgres are parallel-hash- aggregate territory (next arc SP-Hash-Agg).

• Track K — SP-Cloud-Deploy (2026-05-30, V1 SHIPPED). Production deploy story on top of SP-DX-superior's Dockerfile + ghcr.io push. Three artifacts shipped, each individually load-bearing for first-deploy adoption: (1) a Helm chart at deploy/helm/kesseldb/ — single-pod (replicas:1 + Recreate strategy because the engine is single-writer + PVC is RWO), ServiceAccount + 10 GiB PVC + ClusterIP Service exposing all three wire surfaces (binary 6532, HTTP+WS 6533, PostgreSQL 5432) + Deployment with kessel:1100 non-root + TCP-on-binary liveness + readiness probes + KESSELDB_TOKEN env from a pre-existing Secret (default name kesseldb-token, key token) + 4 CPU / 4 Gi limits matching SP-Hash-Agg's 4-way parallel target. Helm v3.16.3 lint: 0 chart(s) failed. (2) deploy/fly/fly.toml + deploy/fly/README.md — Fly.io single-VM deployment pinned to the ghcr.io image, three [[services]] TCP stanzas (one per wire surface), auto_stop_machines=off + min_machines_running=1 (stateful engine — autostop would break long-lived connections), strategy=immediate (single-attach volume). TOML well-formed (Python tomllib parser pass). (3) USAGE §11 + README Deploy section + kind-verify transcript file. Verified end-to-end on vulcan (kind v0.24.0 + Kubernetes v1.31.0 + helm v3.16.3 — all installed user-local to vulcan): helm lint 0 failed → helm template renders 4 K8s objects correctly (SA + PVC + Svc + Deploy; open-mode branch verified via --set auth.secretName='') → kind create cluster → kubectl create secret generic kesseldb-token → helm install kesseldb ./deploy/helm/kesseldb → image side- loaded (GHCR package currently private; documented as a follow-up, see Caveats below) → kubectl rollout status GREEN → kubectl exec deploy/kesseldb -- kessel ... CREATE TABLE / INSERT / SELECT SUM(v) returns = 42 (binary protocol round- trip GREEN) → HTTP /v1/health returns {"status":"ok","primary":true,"view":0,"op_number":4,...} → HTTP /v1/sql SELECT * FROM smoke returns {"status":"ok","bytes":36} (4-byte LE len prefix + 32-byte encoded row). USAGE.md §11 inserted with sub-sections 11.1 (Docker single-host) / 11.2 (Helm) / 11.3 (Fly) / 11.4 (Custom — Nomad/ECS/Cloud Run/systemd-nspawn); former §11-13 (Backup / Wire / Troubleshooting) renumbered to §12-14. README gains a 4-row Deploy table pointing at each artifact. V1 caveats (named, not vague): single-pod / single-VM by design (the named follow-up arc SP-Cloud-Cluster will ship StatefulSet + per-replica PVCs + headless Service + ClusterClient endpoints); no public TLS in the v1 ghcr.io image (--features tls is opt-in; pair with ingress + cert-manager / fly certs if HTTPS is required on the HTTP gateway); GHCR package visibility currently private (default for new ghcr packages; flip to Public in the GitHub UI for one-command kubernetes pull). Zero Rust code touched (this slice is YAML + Markdown only); workspace test count unchanged; default cargo build byte-identical; HTTP/1.1 + WS + binary + PG-wire surfaces byte-untouched; #![forbid(unsafe_code)] honored (no Rust changes); zero new external deps. Six commits: e3eca27 (T1 Helm chart skeleton), 449929d (T2 fly.toml + Fly README), 1a7ceb9 (T3 kind verify transcript), a3b7d0f (T4 USAGE §11), 4c5e793 (T5 README Deploy section), plus this commit (T6 STATUS + progress tracker). Progress tracker docs/superpowers/specs/2026-05-30-kesseldb-spclouddeploy-progress.md CLOSED. Arc closed — TaskList #346 ready for completion.