Re: Question over Incremental Snapshot vs Full Snapshot in rocksDb state backend

2018-10-28 Thread chandan prakash
Thanks a lot Andrey . Your answer to 1st question clarifies well for both the questions. Appreciate your help in clearing doubt Regards, Chandan On Thu, Oct 25, 2018 at 6:10 PM Andrey Zagrebin wrote: > Hi Chandan, > > > 1. Why did we took 2 different approaches using different RocksDB apis ? >

Re: Question over Incremental Snapshot vs Full Snapshot in rocksDb state backend

2018-10-25 Thread Andrey Zagrebin
Hi Chandan, > 1. Why did we took 2 different approaches using different RocksDB apis ? > We could have used Checkpoint api of RocksDB for fullSnapshot as well . The reason here is partially historical. Full snapshot in RocksDB backend was implemented before incremental and rescaling for incremen

Re: Question over Incremental Snapshot vs Full Snapshot in rocksDb state backend

2018-10-24 Thread chandan prakash
Thanks Tzu-Li for redirecting. Would also like to be corrected if my any inference from the code is incorrect or incomplete. I am sure it will help to clear doubts of more developers like me :) Thanks in advance. Regards, Chandan On Wed, Oct 24, 2018 at 9:19 PM Tzu-Li (Gordon) Tai wrote: > Hi

Re: Question over Incremental Snapshot vs Full Snapshot in rocksDb state backend

2018-10-24 Thread Tzu-Li (Gordon) Tai
Hi, I’m forwarding this question to Stefan (cc’ed). He would most likely be able to answer your question, as he has done substantial work in the RocksDB state backends. Cheers, Gordon On 24 October 2018 at 8:47:24 PM, chandan prakash (chandanbaran...@gmail.com) wrote: Hi, I am new to Flink.

Question over Incremental Snapshot vs Full Snapshot in rocksDb state backend

2018-10-24 Thread chandan prakash
Hi, I am new to Flink. Was looking into the code to understand how Flink does FullSnapshot and Incremental Snapshot using RocksDB What I understood: 1. *For full snapshot, we call RocksDb snapshot api* which basically an iterator handle to the entries in RocksDB instance. We iterate over every ent