Hi Artsem, I think it totally makes sense to have a catalog for the Schema Registry. It is also good to hear you want to contribute that. There is few important things to consider though:
1. The Catalog interface is currently under rework. You make take a look at the corresponding FLIP-30[1], and also have a look at the first PR that introduces the basic interfaces[2]. I think it would be worth to already consider those changes. I cc Xuefu who is participating in the efforts of Catalog integration. 2. There is still ongoing discussion about what properties should we store for streaming tables and how. I think this might affect (but maybe doesn't have to) the design of the Catalog.[3] I cc Timo who might give more insights if those should be blocking for the work around this Catalog. Best, Dawid [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-30%3A+Unified+Catalog+APIs [2] https://github.com/apache/flink/pull/8007 [3] https://docs.google.com/document/d/1Yaxp1UJUFW-peGLt8EIidwKIZEWrrA-pznWLuvaH39Y/edit#heading=h.egn858cgizao On 16/04/2019 17:35, Artsem Semianenka wrote: > Hi guys! > > I'm working on External Catalog for Confluent Kafka. The main idea to > register the external catalog which provides the list of Kafka topics and > execute SQL queries like : > Select * form kafka.topic_name > > I'm going to receive the table schema from Confluent schema registry. The > main disadvantage is: we should have the topic name with the same name > (prefix and postfix are accepted ) as this schema subject in Schema > Registry. > For example : > topic: test-topic-prod > schema subject: test-topic > > I would like to contribute this solution into the main Flink branch and > would like to discuss the pros and cons of this approach. > > Best regards, > Artsem >
signature.asc
Description: OpenPGP digital signature
