This is an automated email from the ASF dual-hosted git repository. hubcio pushed a commit to branch feat/message-bus-transports in repository https://gitbox.apache.org/repos/asf/iggy.git
commit e444085a3e1c0838752832951e9ce8c704125b42 Author: Hubert Gruszecki <[email protected]> AuthorDate: Fri Apr 24 11:46:47 2026 +0200 docs(message_bus): name the plane split + TCP-forever invariant (P0-T4) Add a crate-level doc comment to lib.rs covering the replica plane / SDK-client plane distinction, the authenticated-handshake envelope, and the four invariants a module editor has to preserve: fast-path Ready return, writev batching, TCP-only fd delegation, and the 0-RTT-off default on any future QUIC path. Points at the transport-plan notes under Documents/silverhand/iggy/message_bus for full context. Promote the "replica plane = TCP forever" rationale from a prose paragraph at the top of replica_listener.rs to a first-class section so a contributor touching the listener sees the VSR / fd-delegation / writev / RTT reasoning immediately, not as implementation detail. Refs: Documents/silverhand/iggy/message_bus/transport-plan/ (P0-T4, INVARIANTS.md I1). --- core/message_bus/src/lib.rs | 44 ++++++++++++++++++++++++++++++++ core/message_bus/src/replica_listener.rs | 14 ++++++++++ 2 files changed, 58 insertions(+) diff --git a/core/message_bus/src/lib.rs b/core/message_bus/src/lib.rs index 99c76633a..daf52fd83 100644 --- a/core/message_bus/src/lib.rs +++ b/core/message_bus/src/lib.rs @@ -15,6 +15,50 @@ // specific language governing permissions and limitations // under the License. +//! Shard-local message bus with two wire planes. +//! +//! # Plane split +//! +//! - **Replica plane (TCP forever)**: VSR consensus traffic between +//! replicas. Implemented in [`replica_listener`], [`connector`], and +//! [`replica_io`]. Datagram or gateway-terminated transports are NOT +//! supported here and never will be — see +//! `replica_listener`'s module docs for the rationale. +//! - **SDK-client plane**: ephemeral client connections. TCP today via +//! [`client_listener`]; Phase 2+ adds WebSocket / QUIC on a +//! demand-gated basis. At-most-once semantics come from the +//! `(client: u128, request: u64)` pair in `RequestHeader`; the dedup +//! store lives in `core/server-ng/src/dedup.rs` on each shard. +//! +//! # Authenticated handshake +//! +//! Both planes use [`auth::TokenSource`] for identity. The replica +//! plane ships with [`auth::StaticSharedSecret`] (all replicas share +//! one 32-byte key); the client plane will grow per-client token +//! sources in Phase 2+. The envelope lives in +//! `GenericHeader.reserved_command[0..57]` — 32 B BLAKE3 MAC, 8 B +//! timestamp, 16 B nonce, 1 B kind — so the 256 B header layout is +//! unchanged. +//! +//! # Invariants worth naming +//! +//! - [`send_to_client`](IggyMessageBus::send_to_client) and +//! [`send_to_replica`](IggyMessageBus::send_to_replica) return +//! `Ready` on first poll. Consensus code relies on this for +//! reentrancy reasoning; any `.await` in the body breaks it. +//! - [`writer_task`] coalesces up to `max_batch = 256` +//! `Frozen<MESSAGE_ALIGN>` into one `write_vectored_all`. Don't +//! introduce per-message syscalls or per-message encryption. +//! - fd-delegation ([`fd_transfer`]) is TCP-only. TLS / QUIC +//! connections have no dupable plaintext fd, so shard 0 terminates +//! and forwards `Frozen<MESSAGE_ALIGN>` over the existing +//! inter-shard flume. +//! - 0-RTT stays disabled by default on any future QUIC path. Per- +//! command opt-in requires a checked-in idempotence audit. +//! +//! The full invariant list and the transport-plan design notes live +//! under `Documents/silverhand/iggy/message_bus/transport-plan/`. + pub mod auth; pub mod cache; pub mod client_listener; diff --git a/core/message_bus/src/replica_listener.rs b/core/message_bus/src/replica_listener.rs index d2d82d586..b056f30ea 100644 --- a/core/message_bus/src/replica_listener.rs +++ b/core/message_bus/src/replica_listener.rs @@ -17,6 +17,20 @@ //! Inbound TCP listener for replica-to-replica consensus traffic. //! +//! # Replica plane = TCP forever (load-bearing invariant) +//! +//! This module is TCP-only by design, not by omission. Do NOT add WS, +//! QUIC, or HTTP paths here. The VSR chain-hash safety proof +//! (`last_prepare_checksum` in `core/consensus/src/plane_helpers.rs`), +//! the fd-delegation path (`F_DUPFD_CLOEXEC` needs a dupable plaintext +//! fd), the `writev` batching in `writer_task.rs`, and the single-digit +//! RTT assumptions used by the view-change timer all rest on a FIFO +//! byte stream between a bounded set of replicas on a trusted LAN. A +//! datagram-oriented or gateway-terminated transport violates one or +//! more of those assumptions. See +//! `Documents/silverhand/iggy/message_bus/transport-plan/INVARIANTS.md` +//! (I1) for the debate context. +//! //! Runs only on shard 0. On every successful `Ping` handshake the listener //! hands the accepted `TcpStream` to an `on_accepted` callback provided by //! the shard bootstrap, which dup-and-ships the fd to the owning shard via
