Repository navigation
Support FlatBuffers RPC over URMA - #101
Draft
Spicy-cream wants to merge 2 commits into
Draft
Spicy-cream wants to merge 2 commits into
Spicy-cream wants to merge 2 commits into
Conversation
* Add FlatBuffers RPC protocol support Add the optional fb_rpc transport. A frame is a 12-byte FRPC header and fixed request or response metadata, all in network byte order, followed by the FlatBuffers message and an optional attachment. Methods are addressed by stable IDs. Integrate it with Channel, Controller and Server without changing the existing protobuf Protocol callback signatures: - Channel, Controller and Server keep the same layout whether BRPC_WITH_FLATBUFFERS is on or off. Generated stubs reach a brpc::Channel through a library adapter, so `Echo::Stub stub(&channel);` works unchanged. A layout probe built in both configurations checks it. - FlatBuffers methods use Server::MethodProperty, so bvar exposure, concurrency limits, MaxConcurrencyOf() and IgnoreEovercrowdedOf() share one code path. They stay out of the map other protocols search by name. ListFlatBuffersServices() lists them and /status shows them. - A response is accepted only on the socket used by the attempt it answers. - New server connections try fb_rpc right after baidu_std. Protocol IDs and handler indices stay as they are; InputMessenger keeps a separate probe order instead. Keep request buffers immutable across retries and concurrent calls, and manage service ownership, asynchronous completion and concurrency limits. Reject invalid frames and unsupported authentication, compression, checksums and streams. FlatBuffers support requires C++17. CMake and config_brpc.sh switch to C++17 when it is enabled, and the FlatBuffers headers reject older standards, so over-aligned objects use aligned new in the library and in every unit that includes them. Protobuf-only builds are unchanged. The feature stays disabled by default in CMake, Make and Bazel. Unit tests that registered a private protocol as ID 30, now used by fb_rpc, move to ID 127. The codegen test lists the selected Protobuf include directories before shared prefixes. Add 43 protocol tests covering the wire format, service lifecycle, registry, malformed input and real RPC round trips. * Add a generated FlatBuffers RPC example Add a client/server example built from a generated service. By default it runs a bounded loopback smoke test: 13 verified replies, 2 expected schema rejections and a clean shutdown. An opt-in timed mode compares fb_rpc with an equivalent protobuf service over baidu_std on the same port. --duration_s and --warmup_s run closed-loop callers and print QPS, latency percentiles and client CPU as one JSON line; --protocol, --use_bthread and --allow_remote select the protocol, bthread callers and remote endpoints. The measured window and its CPU sample start and end exactly where counting starts and stops, so calls still in flight when the timer expires do not stretch it. The smoke test checks this against a server started with --reply_delay_ms. Request and reply builders reserve the payload size, so large messages are not grown by repeated reallocate-and-copy. The example takes an installed FlatBuffers-enabled bRPC, writes the code generated by flatc and brpc_flatc into its own build directory, and lists the selected Protobuf include directories before shared prefixes. * Add explicit FlatBuffers ON validation Add a runner that builds and tests the FlatBuffers-enabled configuration with CMake, Make or Bazel, rejects empty, filtered or skipped test runs, and compares the class layout probes built with FlatBuffers on and off. Add its self-tests and a GitHub Actions workflow that runs it on Linux and macOS when FlatBuffers-related paths change. The jobs build gperftools as ci-linux.yml does. On macOS they use protobuf@29 followed by the shared Homebrew prefix, as ci-macos.yml does. * Document FlatBuffers setup and example checks Add Chinese and English guides covering how to enable FlatBuffers, the dependencies and the C++17 requirement, building the runtime, running the example with its smoke test and timed comparison, the network RPC API, builder sizing for large messages and the FRPC wire format. Update the README files. * Fail timed benchmark when a caller cannot start RunTimed ignored worker start failures and could exit 0 with fewer callers than requested. Exit 1 in that case, report started_threads, and cover it in smoke.py with a test-only --fail_worker_start_at flag. * Fix latent issues in the FlatBuffers benchmark example - Keep timed-mode latency in fixed-size per-caller histograms instead of one sample per call, so memory no longer grows with duration_s. - Time only the RPC, and append the attachment IOBuf by reference and compare it without copying. - Reject timed runs over short connections, which exhaust local ports. - Close the window early and exit 1 on SIGINT/SIGTERM. - Do not verify the request schema twice on the fb_rpc server. - Accept EINTERNAL as well as -1 for schema rejection. - Document server lifetime, max_concurrency and payload_mib_per_s. * Drop the C++14 rejection check from codegen acceptance FlatBuffers support requires C++17, so the codegen CI does not need to cover older standards. The headers still #error below C++17. --------- Co-authored-by: xulei25 <xulei25@baidu.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: N/A
Problem Summary:
bRPC supports FlatBuffers RPC (
fb_rpc) and URMA transport independently.URMA currently accepts only
baidu_std. Allowingfb_rpcthrough theprotocol check also requires handling large contiguous FlatBuffers allocations
and other IOBuf ranges outside registered memory.
This draft brings in Apache bRPC #3559
as a separate prerequisite commit. The URMA integration is the follow-up
commit
9f4f3621.What is changed and the side effects?
Changed:
fb_rpcin the URMA protocol allowlist when FlatBuffers is compiled in.ranges into registered pool buffers in bounded chunks, and hold those
buffers in the send queue until completion.
--use_urmato bothbenchmark_fbexecutables and a verbose log foraccepted URMA data SENDs to identify the data path in hardware smoke runs.
Side effects:
pool buffers until completion. A fragmented or unaligned FlatBuffers
payload may still require a receive-side copy. Registered send ranges
retain their direct path. Hardware latency and throughput are pending.
are unchanged. The shared URMA send path now stages unregistered ranges
for both supported protocols. If registered staging space is unavailable,
the send fails with
ENOMEM.Validation
WITH_URMA=ON,URMA_USE_MOCK=ONandWITH_FLATBUFFERS=ON:brpc_urma_unittest,brpc_flatbuffers_unittestandbrpc_flatbuffers_protocol_unittestpassed.with an 8,193-byte attachment and 4 KiB pool buffers. It checks that the
request starts in unregistered memory and the server connection reaches
ESTABLISHED. Five consecutive repetitions passed.brpc_urma_unittest.benchmark_fbclient and server built and passed the TCPsmoke test.
hosts, confirm data SENDs on both peers, and compare
fb_rpcURMA/TCPthroughput and latency at the same payload sizes and concurrency.
Check List