shengminw opened a new issue, #4909:
URL: https://github.com/apache/rocketmq/issues/4909

   ## Problem View
   When we set **CONSUME_FROM_LAST_OFFSET** to pull data, if it is the first 
time to start the consumer group, data may be pulled from the 0L. This is not 
what we expected.
   I built a demo to reproduce this situation.
   First sent the data using the producer:
   
![image](https://user-images.githubusercontent.com/107378995/187054813-dfd00802-bb5c-4d17-b2ea-aea2a563cda1.png)
   
   Set consumer mode as **CONSUME_FROM_LAST_OFFSET**.
   
![image](https://user-images.githubusercontent.com/107378995/187054823-193f540e-8669-474b-a520-2d057400fe0d.png)
   
   At the first startup, the consumption situation obtained should 
theoretically not pull the previous message, but the message is still pulled 
from the beginning:
   
![image](https://user-images.githubusercontent.com/107378995/187054840-bba0f765-22e0-400a-986d-1d5eb5e233ed.png)
   
   
   ## Location the problem
   When a consumer group needs to pull messages, it obtains the initial 
location to be consumed according to the startup mode.
   
![image](https://user-images.githubusercontent.com/107378995/187054864-7c409a2e-a9ae-43a1-b3b6-bf5f39e687fd.png)
   
   In cluster mode, the offset is read from the broker side. (cluster mode has 
the above problem)
   
   In broadcast mode, the offset of the local record is read, and -1 is 
returned if it is not read.
   
   In broker, in  **ConsumerManageProcessor#queryConsumerOffset**, the broker 
will query the consumeoffset by consumegroup, topic and queueId.
   
   ● If the offsetTable records the consumption information of this consumer 
group under this topic and this queueId, return this recorded consumeoffset
   ● If there is no record in the offsetTable, the minimum offset in the 
consumequeue will be obtained. (When the minimum offset is less than or equal 
to 0 && offset 0 is in memory, it returns to 0) -> This leads that when 
consumequeue does not clean up expired files, the minimum offset is 0 and it is 
in memory , 0 will be returned, and the client receives an offset of 0, so it 
starts to consume from the starting position.
   
![image](https://user-images.githubusercontent.com/107378995/187054882-ed370b1b-1263-483f-af62-60fe0013e9b2.png)
   
   
   ## Solution
   When querying consumeoffset, set ZeroIfNotFound false in the requestHeader. 
   So, when the previous record is not obtained, it will no longer return 0.
   
![image](https://user-images.githubusercontent.com/107378995/187054887-01f72caa-7b8b-4870-a698-1cc0b90bf223.png)
   
   
   ## Another Question
   
   The above solution can solve this problem, but I still have some doubts 
about the original implementation method
   
   In  **ConsumerManageProcessor#queryConsumerOffset**,why we need to set 
offset as 0L, when minOffset <= 0 and offset 0L is in memory. Can we return 
response just setCode as ResponseCode.QUERY_NOT_FOUND?
   
   
![image](https://user-images.githubusercontent.com/107378995/187054939-96d3e099-be62-401c-ac47-a8819dfa817e.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to