Please note that SourceFunction API is deprecated and is due to be removed, possibly in the next major version of Flink. Ideally you should not be manually spawning threads in your Flink applications. Typically you would only perform data fetching in the sources and do processing in the subsequent operators which you can scale independently from the source parallelism. Can you describe what you are trying to achieve?
Best, Alexander Fedulov On Tue, 31 Oct 2023 at 07:22, Kamal Mittal via user <user@flink.apache.org> wrote: > Hello Community, > > > > I need to have a custom parallel data source (Flink ParallelSourceFunction) > for fetching data based on some custom logic. In this source function, > opening multiple threads via java thread pool to distribute work further. > > > > These threads share Flink provided ‘SourceContext’ and collect records via > source_context.collect() method. > > > > Is it ok to share source context in separate threads and get data? > > > > Is there any issue for downstream operators due to above design? > > > > Rgds, > > Kamal >
