[I] [Bug] CommitLog topicQueueLock.lock(topicQueueKey) 锁粒度问题，导致defaultMessageStore.assignOffset(msg) 中 给LMQ消息分配的INNER_MULTI_QUEUE_OFFSET重复，导致LMQ消费时，LMQ消息丢失 [rocketmq]

Fri, 27 Oct 2023 02:28:30 -0700 


lzkisok opened a new issue, #7511:
URL: https://github.com/apache/rocketmq/issues/7511

   ### Before Creating the Bug Report
   
   - [X] I found a bug, not just asking a question, which should be created in 
[GitHub Discussions](https://github.com/apache/rocketmq/discussions).
   
   - [X] I have searched the [GitHub 
Issues](https://github.com/apache/rocketmq/issues) and [GitHub 
Discussions](https://github.com/apache/rocketmq/discussions)  of this 
repository and believe that this is not a duplicate.
   
   - [X] I have confirmed that this bug belongs to the current repository, not 
other repositories of RocketMQ.
   
   
   ### Runtime platform environment
   
   Centos 7
   
   ### RocketMQ version
   
   RMQ 5.1.4
   
   ### JDK Version
   
   JDk 1.8
   
   ### Describe the Bug
   
   
1，当并发发送两条MQ消息，设置相同的INNER_MULTI_DISPATCH，相当于DefaultMQProducer并发发送了两条同一个LMQ的消息（消息的LMQ相同，内容不同）。
   2，当这两条消息发送到两个的MessageQuene 
时，消息的queneId不同，topicQueueKey是topic-queneId，topicQueueLock.lock(topicQueueKey) 
是锁各自queneId的，所以两条消息是并发处理的。
   3，在消息并发处理时，defaultMessageStore.assignOffset(msg) 
会同时从QueueOffsetOperator拿到LMQ的queneOffste，这时并发拿到的queneOffst相同，导致两条LMQ消息的INNER_MULTI_QUEUE_OFFSET
 相同，LMQ 消息的offset重复
   
   ### Steps to Reproduce
   
   1，DefaultMQProducer并发发送两条相同LMQ的消息（消息的LMQ相同，内容不同）如：
   String secondTopic = "/r7";
   setLmq(msg, new HashSet<>(Arrays.asList(TopicUtils.wrapLmq(firstTopic, 
secondTopic))));
   2. 观察RMQ 日志，确定两个消息的MessageQuene不同。
   3. mqtt客户端消费，只消费到一条。
   
   ### What Did You Expect to See?
   
   在消息并发，且MessageQuene不同时，能正确设置LMQ消息的INNER_MULTI_QUEUE_OFFSET
   
   ### What Did You See Instead?
   
   两条LMQ消息的INNER_MULTI_QUEUE_OFFSET 相同，LMQ 消息的offset重复
   
   ### Additional Context
   
   
![image](https://github.com/apache/rocketmq/assets/24490175/276df455-f274-4155-9350-4a7d1ca6ba34)
   如上图：两条消息的INNER_MULTI_QUEUE_OFFSET相同
   
![image](https://github.com/apache/rocketmq/assets/24490175/27e491d9-8277-48f2-870f-bd172a6da371)
   如上图：在DefaultMQProducer并发发送两条相同LMQ的消息（消息的LMQ相同，内容不同），这里的锁，对LMQ不适用


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to