Hi Mingyu > In the exclusive consumption mode of exclusive, failover, reader, etc. If I understand correctly, your concern is not about read-only partitions, but about being able to read newly sent messages as quickly as possible. Maybe this problem can be solved by adding a new configuration that controls the producer routing. If this configuration is enabled, all newly created or reconnected producers will preferentially send messages to the partition with the least backlog. Is this the result you were expecting? Sincerely, Xiangying
On Mon, Aug 29, 2022 at 1:51 PM Anon Hxy <anonhx...@gmail.com> wrote: > Hi Mingyu > > > expanding partitions or modifying consumption logic cannot reduce the > consumption of new messages in time > > If I understand correctly, we will no need message order guarantee if > applied this feature. If so, I think that expanding partitions can alse > solve the problem: > Consumer and producer will listen the topic partition increase event > if we set `ConsumerConfigurationData#autoUpdatePartitions` as true. When > expanded the partitions, new consumers will be created to consume the > extended partitions, and producers will also send messages to the extended > partitions. The differents is that you want only send messages the > extended partitions but not the accumulated partitions. > > So It seems that we only need implement a costom MessageRouter in the > producer will solve the same problem. > > Thank, > Xiaoyu Hou > > > baomingyu_5 <baomingy...@163.com> 于2022年8月25日周四 10:37写道： > > > Hi all, > > > > In the exclusive consumption mode of exclusive, failover, reader, etc. if > > a large number of messages accumulate due to limited consumption capacity > > or consumption logic problems, expanding partitions or modifying > > consumption logic cannot reduce the consumption of new messages in time. > > Influence, a certain proportion of newly added messages will still be > > produced in the accumulated paritition, and new messages cannot be > quickly > > consumed by restarted consumers or new consumers. In this way, the time > > affected by the accumulation factor may very long. > > > > In order to reduce the impact on the business, through the control of > read > > and write permissions, configure some partitions as read-only, so that > > newly produced messages can be sent to the partitions with less > > accumulation or newly expanded, reducing the impact on the business > caused > > by the accumulation of messages. Reduce the overall time-consuming of > > production--consumption. >
