chenBright commented on code in PR #3144:
URL: https://github.com/apache/brpc/pull/3144#discussion_r4545478491
##########
src/brpc/rdma_transport.cpp:
##########
@@ -30,11 +30,26 @@ DECLARE_bool(usercode_in_pthread);
extern SocketVarsCollector *g_vars;
-void RdmaTransport::Init(Socket *socket, const SocketOptions &options) {
+void RdmaTransport::DoInit(Socket *socket, const SocketOptions &options, bool
use_gdr) {
CHECK(_rdma_ep == nullptr);
- if (options.socket_mode == SOCKET_MODE_RDMA) {
- _rdma_ep = new rdma::RdmaEndpoint(socket);
- _rdma_state = RDMA_UNKNOWN;
+ // gdr mode is a special mode of rdma mode.
+ // both rdma mode and gdr mode need init rdma::RdmaEndpoint.
+ if (options.socket_mode == SOCKET_MODE_RDMA ||
+ options.socket_mode == SOCKET_MODE_GDR) {
+ _rdma_ep = new(std::nothrow)rdma::RdmaEndpoint(socket, use_gdr);
+ if (!_rdma_ep) {
+ // Do NOT call socket->SetFailed() here: the Socket still only
+ // holds its creation reference at this point, so SetFailed()
+ // would synchronously recycle it while OnCreated()/DoInit() are
+ // still initializing it. Just log and fall back to TCP.
+ PLOG(ERROR) << "Fail to create RdmaEndpoint, "
+ "fall back to TCP transport";
+ // No endpoint: later code paths must not assume RDMA is active
+ // and dereference _rdma_ep. Mark the transport as not using RDMA.
+ _rdma_state = RDMA_OFF;
Review Comment:
If endpoint allocation fails, setting `RDMA_OFF` alone is not enough to
safely fall back to TCP: the server's `socket_mode` remains RDMA/GDR, so
`ExecuteServerHandshake()` enters the actual handshake and triggers `CHECK(ep
!= nullptr)`; the client's `Connect()` also still returns `RdmaConnect`, whose
`StartConnect()` likewise checks for a non-null endpoint. This new failure
branch will therefore still terminate the process. Please also switch the
socket mode to TCP and make `Connect()` use the TCP connection logic when there
is no endpoint; add a regression test for endpoint allocation failure on both
sides.
##########
src/brpc/policy/baidu_rpc_protocol.cpp:
##########
@@ -823,8 +842,27 @@ void ProcessRpcRequest(InputMessageBase* msg_base) {
butil::IOBuf req_buf;
int body_without_attachment_size = req_size -
meta.attachment_size();
- msg->payload.cutn(&req_buf, body_without_attachment_size);
+#if BRPC_WITH_GDR
+ bool is_gpu_memory = msg->payload.is_gpu_memory();
+ if (is_gpu_memory) {
+ if (!FillReqBufGpu(&req_buf, msg.get(),
body_without_attachment_size)) {
Review Comment:
The GPU body conversion here only covers the ordinary service branch; the
earlier `IsBaiduMasterService()` branch still directly uses `cutn()` to move
the GPU payload into `SerializedRequest::serialized_data()`. Therefore, after a
GDR request reaches a server configured with `baidu_master_service`, even
without an attachment, if the proxy forwards the SerializedRequest unchanged
over a regular TCP Channel, `SerializeRpcRequest()` still only shares those
device memory blocks, and `writev()` ultimately attempts to send from device
addresses and fails with `EFAULT`; directly parsing the body also causes an
invalid memory access. Consider converting the body to a host IOBuf in the
master-service branch as well, or moving the conversion to a location shared by
both branches; attachments can remain on the GPU.
##########
src/brpc/policy/baidu_rpc_protocol_gpu.cpp:
##########
@@ -0,0 +1,338 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#if BRPC_WITH_GDR
+
+#include <google/protobuf/descriptor.h> // MethodDescriptor
+#include <google/protobuf/message.h> // Message
+#include <google/protobuf/io/zero_copy_stream_impl_lite.h>
+#include <google/protobuf/io/coded_stream.h>
+#include <google/protobuf/text_format.h>
+
+#include "butil/gpu/gpu_block_pool.h"
+#include "butil/iobuf.h" // butil::IOBuf
+#include "butil/logging.h" // LOG()
+#include "butil/memory/scope_guard.h"
+#include "butil/raw_pack.h" // RawPacker RawUnpacker
+#include "butil/strings/string_util.h"
+
+#include "json2pb/json_to_pb.h"
+#include "json2pb/pb_to_json.h"
+#include "brpc/controller.h" // Controller
+#include "brpc/socket.h" // Socket
+#include "brpc/server.h" // Server
+#include "brpc/span.h"
+#include "brpc/compress.h" // ParseFromCompressedData
+#include "brpc/checksum.h"
+#include "brpc/stream_impl.h"
+#include "brpc/rpc_dump.h" // SampledRequest
+#include "brpc/rpc_pb_message_factory.h"
+#include "brpc/policy/baidu_rpc_meta.pb.h" // RpcRequestMeta
+#include "brpc/policy/baidu_rpc_protocol.h"
+#include "brpc/policy/most_common_message.h"
+#include "brpc/policy/streaming_rpc_protocol.h"
+#include "brpc/details/usercode_backup_pool.h"
+#include "brpc/details/controller_private_accessor.h"
+#include "brpc/details/server_private_accessor.h"
+
+namespace brpc {
+namespace policy {
+
+// Notes:
+// 1. 12-byte header [PRPC][body_size][meta_size]
+// 2. body_size and meta_size are in network byte order
+// 3. Use service->full_name() + method_name to specify the method to call
+// 4. `attachment_size' is set iff request/response has attachment
+// 5. Not supported: chunk_info
+
+// Pack header into `buf'
+
+const int header_size = 12;
+// if we recv data into gpu, the header/meta/body will be copied to cpu and
processed.
+// in to to limit the count of d2h, we will prefetch some bytes from gpu to
cpu.
+// if header_size + meta_size + body_size(without attachment) is less than the
+// prefetch size, then one d2h is enough for one rpc.
+// The prefetch size is derived from the CPU block pool's block size so that
the
+// allocation never exceeds the allocator's single-block limit and stays
effective
+// if the block size changes.
+
+static const int kDefaultPrefetchD2HSize = 512;
+
+static int GetPrefetchD2HSize(butil::gdr::BlockPoolAllocator* host_allocator) {
+ const int64_t block_size = host_allocator->get_block_size();
+ return block_size > 0 ? static_cast<int>(block_size)
+ : kDefaultPrefetchD2HSize;
+}
+
+ParseResult ParseRpcMessageGpu(butil::IOBuf* source, Socket* socket,
+ bool /*read_eof*/, const void*) {
+
+ char header_buf[12];
+ size_t n = 0;
+ uint32_t body_size;
+ uint32_t meta_size;
+ ParseError pe = PARSE_OK;
+
+ void* prefetch_d2h_data = NULL;
+ bool is_gpu_memory = source->is_gpu_memory();
+ if (!is_gpu_memory) {
+ // A non-GPU buffer here means the message arrived through a non-GDR
+ // path; reject it instead of aborting the whole process.
+ LOG(ERROR) << "RpcMessage is not in gpu!!!";
+ return MakeParseError(PARSE_ERROR_ABSOLUTELY_WRONG);
+ }
+ butil::gdr::BlockPoolAllocator* host_allocator =
butil::gdr::BlockPoolAllocators::GetInstance()->get_cpu_allocator();
+ const int prefetch_d2h_size = GetPrefetchD2HSize(host_allocator);
+ prefetch_d2h_data = host_allocator->AllocateRaw(prefetch_d2h_size);
+ if (prefetch_d2h_data == NULL) {
+ // Pool exhausted or single-block limit hit: ask the caller to retry
+ // instead of crashing the whole server.
+ LOG(ERROR) << "alloc host data for prefetch d2h failed, size="
+ << prefetch_d2h_size;
+ return MakeParseError(PARSE_ERROR_NOT_ENOUGH_DATA);
Review Comment:
When a complete frame has arrived but the host pool is exhausted or
expansion fails, returning `PARSE_ERROR_NOT_ENOUGH_DATA` here does not schedule
a retry: `ProcessNewMessage()` exits the parsing loop, the RDMA receive path
calls it again only when a subsequent receive completion event arrives, and
`DeallocateRaw()` has no wake-up mechanism. Therefore, even if memory becomes
available later, the frame may remain in the receive buffer until the RPC times
out or another receive event occurs. Consider treating allocation failure as a
terminal parsing error and closing the connection, or implementing a
resource-recovery retry that does not depend on new input; do not treat
insufficient resources as an incomplete frame.
##########
BUILD.bazel:
##########
@@ -66,6 +66,11 @@ DEFINES = [
}) + select({
"//bazel/config:brpc_with_rdma": ["BRPC_WITH_RDMA=1"],
"//conditions:default": [],
+}) + select({
+ # NOTE: `defines` is auto-prefixed with -D by Bazel, so do NOT add -D here.
+ # Compare with the rdma/glog/thrift entries above which also omit the
prefix.
+ "//bazel/config:brpc_with_gdr": ["BRPC_WITH_GDR=1"],
Review Comment:
When only `--define BRPC_WITH_GDR=true` is passed, this does not enable
`BRPC_WITH_RDMA` or select the RDMA `-libverbs` link option. `gdr_transport.h`
declares a class that inherits from `RdmaTransport`, but the latter is excluded
by `#if BRPC_WITH_RDMA`, so compilation fails even when the CUDA and verbs
dependencies are correctly installed. Please make GDR enable the RDMA macro and
link dependency as `config_brpc.sh` does, or explicitly reject this combination
during configuration.
##########
src/brpc/policy/crc32c_checksum.cpp:
##########
@@ -29,10 +29,45 @@ namespace policy {
namespace {
+#if BRPC_WITH_GDR
+// CRC32C is computed on the CPU, so GPU memory cannot be read through CPU
+// pointers. Copy the block to a host buffer first and extend the crc over the
+// copy; the data itself stays in GPU memory (the GPU -> GPU data path of GDR
+// is untouched).
+static uint32_t ExtendCrc32cFromGpu(uint32_t crc, const butil::IOBuf& buf) {
+ const size_t size = buf.size();
+ if (size == 0) {
+ return crc;
+ }
+ char* host = new (std::nothrow) char[size];
+ if (host == nullptr) {
+ LOG(ERROR) << "Fail to alloc host buffer to checksum gpu memory";
+ // Fail closed: return a value that cannot match the expected crc.
+ return crc ^ 0xdeadbeefu;
Review Comment:
`crc ^ 0xdeadbeefu` is still a valid CRC value and cannot indicate checksum
failure. When GPU attachment checksumming is enabled, if allocation of the
temporary host buffer fails on both the sender and receiver, both sides return
the same value based on the same body CRC; `Crc32cCompute()` sends that value,
and `Crc32cVerify()` only compares them for equality, ultimately returning
success without checking the attachment at all. Consider having the internal
calculation function return the success status and CRC separately: return
`false` immediately on verification failure, and abort normal sending through
the Controller/serialization call chain on calculation failure, rather than
substituting a fixed XOR value for the error.
##########
src/brpc/rdma/rdma_endpoint.cpp:
##########
@@ -1907,6 +2002,24 @@ void RdmaEndpoint::DebugInfo(std::ostream& os,
butil::StringPiece connector) con
<< connector << "rdma_read_buf=" << _input_processor.read_buf().size();
}
+int RdmaEndpoint::GlobalGdrInitialize() {
+#if BRPC_WITH_GDR
+ // GetGdrBlockSize() returns 0 on an invalid FLAGS_gdr_block_size_kb;
+ // subtracting the header from it would underflow to a huge value.
+ const size_t gdr_block_size = butil::gdr::GetGdrBlockSize();
+ if (gdr_block_size <= IOBUF_BLOCK_HEADER_LEN) {
Review Comment:
This validation still accepts `--gdr_block_size_kb=1`, but the advertised
receive length is only 992 after subtracting the 32-byte header. Both V2's
`ValidHelloMessage()` and V3's `ValidRdmaHello()` require `block_size >=
MIN_BLOCK_SIZE` (1024), so this configuration passes initialization but causes
all connections to fall back to TCP, making GDR unavailable. Please validate
the handshake minimum against the length after subtracting the header,
explicitly reject this configuration at startup, and cover the 1 KB/2 KB
boundary cases.
##########
src/brpc/rdma/rdma_handshake_server.cpp:
##########
@@ -207,8 +207,14 @@ ParseResult ExecuteServerHandshake(butil::IOBuf* source,
Socket* socket) {
// Only RDMA-mode connections carry a live RdmaEndpoint and run the real
// handshake. A connection that is not in RDMA mode (RDMA compiled in but
// this connection is plain TCP, or RDMA not compiled at all) falls back.
+ // GDR is a special mode of RDMA: it shares the same RDMA handshake.
#if BRPC_WITH_RDMA
- if (socket->socket_mode() == SOCKET_MODE_RDMA) {
+ SocketMode mode = socket->socket_mode();
+ if (mode == SOCKET_MODE_RDMA
+#if BRPC_WITH_GDR
+ || mode == SOCKET_MODE_GDR
Review Comment:
This allows the GDR handshake, but the server does not yet guarantee that
the GPU-aware `baidu_std` parser is registered. For example, with
`socket_mode=SOCKET_MODE_GDR` and `enabled_protocols="http"`, the existing
option validation allows startup, and `BuildAcceptor()` retains the handshake
and HTTP but removes the Baidu RPC parser. After the client completes the
handshake, GPU data passes through `CutInputMessage()` to HTTP's
`ParseFromIOBuf()`, which passes the device address directly to
`http_parser_execute()`, causing an invalid memory access. The termination
guard for invalid frames in `ParseRpcMessageGpu()` does not run in this
configuration. Please reject configurations that do not enable `baidu_std`
during GDR server initialization, or prevent GPU input from reaching non-GPU
parsers at the dispatch boundary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]