>> > 1. Does the consumer returns buffer of the batch message? > 2. If returns a batch message buffer, how do we handle the flow permits? It’s better to clarify in the proposal
It will not change the semantics of the batch-message. Batch-message gets split into individual messages at the consumer side and this feature will make sure that payload of individual messages can be backed by a pooled buffer. So, it doesn't affect the batch message processing path. >> users don’t need to specify the schema when using the RawConsumer since we might occurs schema incompatible issue when using Schema.BYTEBUFFER. ByteBuffer/ByteBuf provides an easy way to access data from the direct-memory. So, ByteBuffer is a suitable return type and RawConsumer doesn't fit into that requirement. Also, it would be better if users can use this feature with the same consumer api and the same access pattern. Therefore, consumer with ByteBuffer-schema will be a more preferable api for the users. >> doc does not tell much about how we have to change the Schema API in order to pass the Consumer configuration to the Schema implementation. ByteBuffer is already supported schema type in the pulsar-client and this feature doesn't require any changes in the Schema API. As it's shown in application-usage <https://github.com/apache/pulsar/wiki/PIP-83-:-Pulsar-client:-Message-consumption-with-pooled-buffer#application-usage> section, users can enable this feature by passing the "poolMessages" flag and existing ByteBuffer schema, and Consumer will create a message with a payload backed by pooled buffer which can be accessed by the ByteBuffer. Consumer<*ByteBuffer*> consumer = client.newConsumer(Schema.*BYTEBUFFER*). *poolMessages* (true).subscriptionName(subscriptionName)topic(topicName).subscribe(); Message<*ByteBuffer*> msg = consumer.receive(); Thanks, Rajan On Mon, Mar 29, 2021 at 11:37 PM Enrico Olivelli <eolive...@gmail.com> wrote: > Rajan (and Matteo), > very interesting feature. > > How are we going to implement this feature ? > it looks like it is bound to the Schema.BYTEBUFFER implementation, the > doc does not tell much about how we have to change the Schema API in > order to pass the Consumer configuration to the Schema implementation. > > Enrico > > Il giorno mar 30 mar 2021 alle ore 06:30 PengHui Li > <codelipeng...@gmail.com> ha scritto: > > > > Nice feature, I just have some questions want to confirm. > > > > 1. Does the consumer returns buffer of the batch message? > > 2. If returns a batch message buffer, how do we handle the flow permits? > It’s better to clarify in the proposal > > > > About the API, my feeling is a RawConsumer is more suitable here, users > don’t need to specify the schema when using the RawConsumer since we might > occurs schema incompatible issue when using Schema.BYTEBUFFER. So > RawConsumer is more clear for this purpose? > > > > Thanks, > > Penghui > > On Mar 30, 2021, 10:48 AM +0800, Rajan Dhabalia <rdhaba...@apache.org>, > wrote: > > > Hello, > > > We have created a PIP-83 for Pulsar client: Message consumption with > pooled > > > buffer. > > > > > > > https://github.com/apache/pulsar/wiki/PIP-83-:-Pulsar-client:-Message-consumption-with-pooled-buffer > > > > > > > > > Thanks, > > > Rajan >
