Re: Questions about implementing a flink source

2021-06-08 Thread Evan Palmer
t; such as record handover, we can also discuss pulling up a common base class > to the async sources and the > SingleThreadMultiplexSourceReaderBase. > >> Ah, okay, this helped a lot. I'd missed that rebalancing dynamically >> would break ordering guarantees, so when I rea

Re: Questions about implementing a flink source

2021-05-07 Thread Evan Palmer
; > [1] > https://github.com/apache/flink/blob/bf40fa5b8cd5b3a8876f37ab4aa7bf06e8b9f666/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSource.java#L75-L75 > [2] > https://github.com/apache/flink/blob/bf40fa5b8cd5b3a8876f37ab4aa7bf06e8b9f6

Questions about implementing a flink source

2021-05-02 Thread Evan Palmer
Hello, I’m new to Flink. I’m trying to write a source for Pub/Sub Lite which is a partition based Pub/Sub product, and I have a few questions. 1. I saw that there are two sets of interfaces used in existing sources: The RichSourceFunction, and the set of interfaces from FLIP-27. It seems like the