Re: StateBackend

2016-02-16 Thread Matthias J. Sax
Thanks for the input. Just to clearly my understanding: by "Flink-embedded [...] scale out as the Flink job scales out", you mean that each TM hosts an embedded state backend service, ie, all those instances form a logically single but distributed backend service? How is is ensure, that the state

Re: StateBackend

2016-02-16 Thread Stephan Ewen
I think this is actually a pretty good question. Right now, there are two different types of state backends: (1) Flink-embedded. They are independent of external services, scale out as the Flink job scales out, and are really mainly a way of storing and backuping key/value state. For exa

Re: StateBackend

2016-02-15 Thread Aljoscha Krettek
Hi, sorry about not answering but I wanted to wait since I already voiced my opinion on the PR. I think it is better to assume an already running redis because it is easier to work around clashes in running redis instances (ports, data directory, and such). Then, however, care needs to be taken

Re: StateBackend

2016-02-15 Thread Matthias J. Sax
Anyone? Otherwise, I will suggest to move forward with the PR using the assumption that Redis must be started manually. -Matthias On 02/11/2016 08:28 PM, Matthias J. Sax wrote: > Hi, > > In Flink it is possible to have different backends for operator state. I > am wondering what the best approa