Re: Managed operator state treating state of all parallel operators as the same

Hi, I think you can do this already, please take a look at OperatorStateStore::getUnionListState(…). You could add state to this only one one parallel operator instance, e.g. by checking the subtask index. Best, Stefan > Am 03.07.2017 um 18:14 schrieb Fabian Hueske : > > Hi Gerard, > > such

Re: Managed operator state treating state of all parallel operators as the same

Hi Gerard, such a type of state is not yet supported, but on the roadmap. The feature is called Broadcast State. A description is in JIRA [1]. Best, Fabian [1] https://issues.apache.org/jira/browse/FLINK-4940 2017-07-03 17:50 GMT+02:00 gerardg : > Hello, > > Is it possible to have managed oper

Managed operator state treating state of all parallel operators as the same

Hello, Is it possible to have managed operator state where all the parallel operators know that they have the same state? Therefore, it would be only snapshotted by one of them and in case of failure (or rescaling) all would use that snapshot. For the rescaling case I guess I could use union redi

Re: Kafka and Flink integration

Hi Urs! Inside Flink (between Flink operators) - Kryo is not a problem, but types must be registered up front for good performance - Tuples and POJOs are often faster than the types that fall back to Kryo Persistent-storage (HDFS, Kafka, ...) - Kryo is not recommended, because its binary da

Re: External checkpoints not getting cleaned up/discarded - potentially causing high load

On Mon, Jul 3, 2017 at 12:02 PM, Stefan Richter wrote: > Another thing that could be really helpful, if possible, can you attach a > profiler/sampling to your job manager and figure out the hotspot methods > where most time is spend? This would be very helpful as a starting point > where the probl

Re: External checkpoints not getting cleaned up/discarded - potentially causing high load

Hi, I have two quick questions about this problem report: 1) Which state backend are you using? 2) In case you are using RocksDB, did you also activate incremental checkpointing when moving to Flink 1.3. Another thing that could be really helpful, if possible, can you attach a profiler/samplin

Re: a lot of connections in state "CLOSE_WAIT"

Hello, is this list constantly growing? If you reload the WebUI do they pile up again? My guess would be the watermarks display is overloading the metrics handler. If i remember correctly the WebUI keeps fetching the watermark metrics regardless of what page you're looking at. Things to inv

unsubscribe

unsubscribe