Make totally sense and is a known issue:
https://issues.apache.org/jira/browse/KAFKA-6711

Please follow up the ticket and/or PR -- not sure what the current
status is.


-Matthias

On 4/17/18 11:16 PM, David Chu wrote:
> I have a custom in-memory state store which I’d like to configure as a global 
> store for my Kafka Streams application.  However, even though my state store 
> is not persistent (i.e. the implementation of StateStore.persistent() returns 
> false) it appears that the GlobalStateManager still continues to write offset 
> checkpoints to disk and then reads those checkpoints upon recovery of the 
> Kafka Streams application to adjust the offsets of the globalConsumer.  
> Because my state store is not persistent though, it does not preserve the 
> previous data which was read from the topic before the streams application 
> was restarted.  Therefore, the fact that the globalConsumer has its offsets 
> set to the values read from the checkpoint file upon initialization causes my 
> state store to lose previously consumed data.  To avoid this situation, it 
> would seem like non-persistent state stores should not have their offsets 
> checkpointed when they are used as a global store; this way the 
> globalConsumer would always consume events from the beginning of the topic.  
> This has the consequence of slowing down start-up time of the stream since it 
> will have to recover more messages from the topic but in my case the number 
> of messages on this topic is not expected to be large and I’m willing to 
> tolerate the slightly slower start-up times.  Therefore, is it possible to 
> disable offset checkpointing for in-memory global stores?  If not, is it 
> reasonable to request such a feature to be added?
> 
> Thanks,
> David 
> 

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to