Re: [DISCUSS] PIP 83 : Pulsar client: Message consumption with pooled buffer

2021-03-30 Thread Matteo Merli
Using explicit release of the message resources, opens up many different ways to optimize the consumption path. Pooling the direct memory when consuming with ByteBuffer schema is just one of them, but definitely not the only one. For example we could also later on: * Pool instances of MessageImp

Re: [DISCUSS] PIP 83 : Pulsar client: Message consumption with pooled buffer

2021-03-30 Thread Rajan Dhabalia
>> > 1. Does the consumer returns buffer of the batch message? > 2. If returns a batch message buffer, how do we handle the flow permits? It’s better to clarify in the proposal It will not change the semantics of the batch-message. Batch-message gets split into individual messages at the consumer

Re: [DISCUSS] PIP 83 : Pulsar client: Message consumption with pooled buffer

2021-03-29 Thread Enrico Olivelli
Rajan (and Matteo), very interesting feature. How are we going to implement this feature ? it looks like it is bound to the Schema.BYTEBUFFER implementation, the doc does not tell much about how we have to change the Schema API in order to pass the Consumer configuration to the Schema implementati

Re: [DISCUSS] PIP 83 : Pulsar client: Message consumption with pooled buffer

2021-03-29 Thread PengHui Li
Nice feature, I just have some questions want to confirm. 1. Does the consumer returns buffer of the batch message? 2. If returns a batch message buffer, how do we handle the flow permits? It’s better to clarify in the proposal About the API, my feeling is a RawConsumer is more suitable here, us

[DISCUSS] PIP 83 : Pulsar client: Message consumption with pooled buffer

2021-03-29 Thread Rajan Dhabalia
Hello, We have created a PIP-83 for Pulsar client: Message consumption with pooled buffer. https://github.com/apache/pulsar/wiki/PIP-83-:-Pulsar-client:-Message-consumption-with-pooled-buffer Thanks, Rajan