Hi Jinsui, Thanks for opening the discussion and creating the FLIP. My main concern isn't so much the connector, it's more that we're talking about yet another Redis connector implementation but we actually don't have a proper Redis connector overall. Ideally it would be great if we could get https://github.com/apache/flink-connector-redis-streams/ in proper shape (perhaps you could help with some reviews) and then moving all Redis connectors into one mono repo, kind of how it's done for AWS connectors.
Best regards, Martijn On Tue, Dec 19, 2023 at 6:25 PM Jinsui Chen <ischenjin...@gmail.com> wrote: > > Hi, there > > We would like to start a discussion thread on "FLIP-404: Create a Redis > HyperLogLog Connector for Flink"[1]. > > There has also been a lot of discussion in the past about Redis connectors, > including creating lookup and sink connectors for generic key-value get and > put, and creating source and sink connectors for streams structures. > > Different data structures are used in different ways. I think we can build > generic connectors based on KVs and create special-purpose connectors for > specific data structures. > > For example, for list structure, end-user can specify the data to be placed > or get on the left or right side of the list to build the sink or source > connector, and for hyperloglog, end-user needs to write the data to > multiple keys by the content of the table field. > > For the differentiated usage of different data structures, a unified > connector may be difficult to maintain due to too much capacity. > > Based on above, this FLIP is proposed. > > Looking forward to any comment or feedback. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-404%3A+Create+a+Redis+HyperLogLog+Connector+for+Flink > > Best regards, > Jinsui
