Dear Kanchi, Thanks for your proposal. I think a similar feature is available in Flink Pulsar Connector. So, from the sql semantics and philosophy there should not be any issues IMHO. Could you please elaborate more on the motivation (e.g., user requests, enabling new use-cases for Kafka, etc) behind introducing the similar feature for Kafka connector?
Regards, Jeyhun On Fri, May 10, 2024 at 8:20 AM Kanchi Masalia <kmasa...@pinterest.com.invalid> wrote: > Dear Flink Community, > > I hope this message finds you well. I am considering the idea of extending > FlinkSQL's capabilities to include direct queries from Kafka topics, > specifically through the use of "SELECT * FROM kafka_topic". Currently, as > we are aware, FlinkSQL permits queries like "SELECT * FROM > flink_sql_table". > > I am writing to seek your valued opinions and insights on whether extending > this capability to include direct Kafka topic queries would align with the > guidelines and architectural philosophy of FlinkSQL. This potential feature > could offer a more streamlined approach for users to access and analyze > data directly from Kafka potentially simplifying workflow processes. > > Here are a few points I would like your input on: > > 1. Does this idea align with the current and future usage philosophy of > FlinkSQL? > 2. Are there potential challenges or conflicts this feature might pose > against existing FlinkSQL guidelines? > 3. Would this enhancement align with how you currently use or plan to > use Flink in your data processing tasks? > > Your feedback will be instrumental in guiding the direction of this > potential feature. > > Thank you very much for considering my inquiry. I am looking forward to > your insights and hope this can spark a constructive discussion within our > community. > > Thanks, > Kanchi Masalia >
