Re: Output[StreamRecord[T]] thread safety

2021-08-03 Thread Caizhi Weng
Hi! It is not documented. If you're interested you can refer to OperatorChain class and StreamTask class.

Re: Output[StreamRecord[T]] thread safety

2021-08-03 Thread Yuval Itzchakov
Hi Caizhi, Thanks for the reply. Is this documented somewhere? On Tue, Aug 3, 2021 at 10:56 AM Caizhi Weng wrote: > Hi! > > As far as I know, this output.collect thingy is not thread safe, and you > should never run your operator's main logic (from reading in the record to > writing the results

Re: Output[StreamRecord[T]] thread safety

2021-08-03 Thread Caizhi Weng
Hi! As far as I know, this output.collect thingy is not thread safe, and you should never run your operator's main logic (from reading in the record to writing the results out) in a separated thread. Flink's runtime expect the whole operator chain to run in a single thread. Yuval Itzchakov 于2021

Output[StreamRecord[T]] thread safety

2021-08-02 Thread Yuval Itzchakov
Hi, Flink 1.13.1 Scala 2.12.4 I have an implementation of an AbstractStreamOperator, where in it's processElement function enqueues an element to a queue which is pooled from a background thread. When processing the elements in the background, I use the Output class to emit elements downstream w