On Fri, Apr 26, 2019 at 1:41 AM vino yang <yanghua1...@gmail.com> wrote:
> You are right, currently, the queryable state has few users. And I totally > agree with you, it makes the streaming works more like a DB. > Alas, I don't think queryable state will really be used much in production other than for ad hoc queries or debugging. Real data stores at scale are resilient, replicated, and with very low downtime. In my opinion, Flink jobs don't sufficiently meet these requirements to work as a replacement for a data store. Jobs too frequently fail and restart because of checkpoint failures, particularly ones with large state. And when a job does restart, all too often local restore can't be used (e.g. if you loose a node). And since there is no fine grained job recovery and there is no hot replicas of the data, all the state will need to be restored from the DFS, which for something with large state can take a while. It's a nice idea, just not realistic in practice.
