flatbuffers support(3/3): Add flatbuffers protocol support for brpc by Q1ngbo · Pull Request #3197 · apache/brpc · GitHub
Skip to content

flatbuffers support(3/3): Add flatbuffers protocol support for brpc - #3197

Open
Q1ngbo wants to merge 1 commit into
apache:masterfrom
Q1ngbo:feat-fb-protocol
Open

Q1ngbo wants to merge 1 commit into
apache:masterfrom
Q1ngbo:feat-fb-protocol

Conversation

@Q1ngbo

@Q1ngbo Q1ngbo commented Jan 15, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: #3062 #3196

Problem Summary:
Hi, 本提交添加了对flatbuffers协议的完整支持。我增加了与“baidu_std”并列的协议,暂时命名为“fb_rpc”,主要实现在文件flatbuffers_protocol.cpp/h中。我认为以下2点值得关注:

  1. 类似于pb协议(即"baidu_std")的CallMethod,fb协议的请求起始于FBCallMethod,但二者十分相似,仅有几行差异,所以我使用函数模板CallMethodInternal提取了二者中的公共部分,使用参数is_pb区分,CallMethod传入参数为true,FBCallMethod为false。较宏观的benchmark测试显示修改后对pb协议几乎没有影响。
  2. Protocol类中的SerializeRequest和PackRequest这两个hook将形参固定为了google::protobuf类型,为了支持fb协议,我将它们改为了void*类型,并对所有协议中的这两个hook进行了适配,在对应函数开头加上了类型转换。

由于该提交依赖于#3196,且编译机器上未安装flatbuffers,所以本pr暂时无法直接编译。我计划根据评审老师的意见再进行修改并添加单测等完整实现。

Check List:

@Q1ngbo

Q1ngbo commented Jan 15, 2026

Copy link
Copy Markdown
Contributor Author

Comment thread src/brpc/policy/flatbuffers_protocol.cpp Outdated
Comment thread src/brpc/policy/flatbuffers_protocol.cpp
Comment thread src/brpc/policy/flatbuffers_protocol.cpp Outdated
Comment thread src/brpc/policy/flatbuffers_protocol.cpp
Comment thread src/brpc/protocol.h Outdated
butil::IOBuf* request_buf,
Controller* cntl,
const google::protobuf::Message* request);
const void* request_obj);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个改动影响太大了,我认为是不可接受的。不仅是brpc代码库中的所有protocol实现需要修改,而且其他用户自己实现的protocol也要修改(即不兼容升级)
我提一个思路你参考一下:你可以让brpc::flatbuffers::Message继承brpc::NonreflectableMessage,这样这个message就自动继承了google::protobuf::Message,这样就可以在不修改brpc框架接口的前提下扩展出flatbuffer协议。你可以参考brpc Redis/Memcached等协议的实现,这些协议实际上并不用到protobuf,但也保持了google::protobuf::Message的接口形式。

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

namespace brpc {
namespace policy {

struct FBRpcRequestMeta {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

有没有考虑过这个Meta也用flatbuffers格式呢?这样以后新增字段不会破坏兼容性

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不太好实现。因为改用fbs定义meta部分后,需要通过FlatBufferBuilder构建meta,赋值时需要走flatbuffers的接口,这会有额外的处理开销。出于性能考虑,当前Meta字段与rpc message被放在了连续的内存中,使用FlatBufferBuilder创建meta时就无法使用该优化了。
也无法直接使用flatc生成的meta struct,因为flatc会将成员变量属性设置为private,且只能通过特定接口访问成员变量。

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

至少在协议中要加上meta的长度,这样以后能增加字段吧。

Key components:
- Protocol Implementation
  * Added flatbuffers_protocol.cpp/h for parsing, serialization and
     request/response handling.
  * Registered "fb_rpc" protocol in global.cpp
- Core Infrastructure Updates
  * Channel: Added FBCallMethod and CallMethodInternal<bool is_pb>
     template for dual pb/fb support.
  * Controller: Added _fb_method, _fb_response fields and
     is_use_flatbuffer() method for flatbuffers state tracking.
  * Server: Added FlatBuffersMethodProperty/ServiceProperty structures
     and AddService() overloads for flatbuffers::Service registration.
- Protocol Handler Refactoring
  * Updated all protocol handlers (baidu_rpc, http, http2, redis, etc.)
     to use generic void* pointers instead of google::protobuf types,
     enabling protocol-agnostic message handling.
@AnDiXL

AnDiXL commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants