[DISCUSS] FLIP-166: Flink/Pinot Connector

Hi all, I'd like to propose a Flink/Pinot connector to solve the problems of streaming/batch unification for the real-time analytical infrastructure on top of Flink and Pinot. Apache Pinot is a real-time distributed OLAP datastore with an inbuilt lambda architectur

Re: Apache Pinot Sink

pares the incremental counter value of > > the checkpointed segment ID with segments that already exist in Pinot for > > the same table and subtask ID. If segments with a higher counter value in > > their IDs are discovered, they are deleted to avoid duplicates. After > that,

Re: Apache Pinot Sink

Hi Mats, Glad to see this interest! We at Uber are also working on a Pinot sink (for BATCH execution), with some help from the Pinot community on abstracting Pinot interfaces for segment writes and catalog retrieval. Perhaps we can collaborate on this proposal/POC. Cheers, Yupeng On Wed, Jan