hubcio opened a new pull request, #3134: URL: https://github.com/apache/iggy/pull/3134
The previous cache/connection.rs was a single-stream blob that held a RefCell<TcpStream> across .await, awaited kernel write completion in the send path, and serialized fan-out through one shared lock. Under VSR pipelining a slow peer stalled sends to every other peer in the same dispatch round, killing parallel quorum collection. Reentrant sends to the same peer would also panic on BorrowMutError. Rebuilt the crate around a per-connection writer task that drains a bounded async-channel mpsc and submits batches via a single write_vectored_all syscall, with the dup(2)-split read half handled by an independent reader task. send_to_* is now sync try_send under the async fn shell: zero awaits, returns SendError::Backpressure on full so VSR can recover via WAL retransmission or view-change timeouts. The lifecycle module owns a root Shutdown / ShutdownToken plus a ConnectionRegistry<K> that tracks the per-peer Sender and both task handles for graceful drain. The directional rule (lower id dials, higher id accepts) eliminates the dialed- both-ways race without a tiebreaker. TCP_NODELAY is set on every socket, and Message::into_frozen() removes the per-send memcpy. Three new integration tests prove the architectural properties: backpressure, vectored batching, head-of-line freedom under a saturated peer. track_background early-returns when the shutdown token is already triggered: shutdown drains the background-task vec exactly once, so a handle pushed after the drain would be leaked. Tasks spawned post-shutdown have already observed the cancellation by the time they reach the tracker, so dropping the handle is the right thing. The compio stream framing helpers live in message_bus::framing (formerly message_bus::codec, renamed to avoid the collision with binary_protocol::codec, which is the sans-IO WireEncode / WireDecode trait module). MESSAGE_ALIGN is no longer duplicated: it is promoted to pub in iggy_binary_protocol::consensus and the trivial message_to_frozen wrapper is dropped in favour of calling Message::into_frozen() at the writer-task call sites directly. Consensus, metadata, partitions, shard, and simulator are updated to the new MessageBus shape; core/server is left untouched as legacy code. Cluster config gains the per-node fields the bus needs to identify replicas (replica_id, tcp_replica port). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
