Re: Ask for SQL using kafka in Flink

2018-06-05 Thread Timo Walther
@Shuyi: Yes, a more advanced table example would be helpful anyway and combining it with Kafka/Avro end-to-end would be even better. @Will: I totally agree that the current connector ecosystem could be improved. This is also on mid-term roadmap. Contributors that could help here are very welco

Re: Ask for SQL using kafka in Flink

2018-06-04 Thread Will Du
Yes, I am also looking for examples for Kafka avro table examples in java and command line. Also, Kafka avro table sink is still missing. In addition, once we have Kafka topic, the API should read the schema directly from schema file or schema registry. The way of current API supporting lacks of

Re: Ask for SQL using kafka in Flink

2018-06-04 Thread Rad Rad
Yes I totally agree Shuyi Chen, if anyone has an example which does the pipeline streaming processing (Kafka-Flink-SQL), it will be useful for all of us. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Ask for SQL using kafka in Flink

2018-06-04 Thread Rad Rad
Thanks Timo. I will go over there. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Ask for SQL using kafka in Flink

2018-06-04 Thread Shuyi Chen
Given the popularity of Flink SQL and Kafka as streaming source, I think we can add some examples of using Kafka[XXX]TableSource in flink-examples/flink-examples-table module. What do you guys think? Cheers Shuyi On Mon, Jun 4, 2018 at 12:57 AM, Timo Walther wrote: > Hi, > > as you can see in c

Re: Ask for SQL using kafka in Flink

2018-06-04 Thread Timo Walther
Hi, as you can see in code [1] Kafka09JsonTableSource takes a TableSchema. You can create table schema from type information see [2]. Regards, Timo [1] https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/k

Re: Ask for SQL using kafka in Flink

2018-06-02 Thread Radhya Sahal
Thanks Rong, I used Flink 1.3.0 in case of using Flink 1.5 how can I define jsonschema? Yes, there are two names but now I put one name only and I want to define jsonschema. Rong Rong wrote > Hi Radhya, > > Can you provide which Flink version you are using? Based on the latest > FLINK 1.5 r

Re: Ask for SQL using kafka in Flink

2018-06-02 Thread Radhya Sahal
Thanks Rong, I used Flink 1.3.0 in case of using Flink 1.5 how can I define jsonschema?Yes, there are two names but now I put one name only and I want to define jsonschema. Rong Rong wrote > Hi Radhya,Can you provide which Flink version you are using? Based on the > latestFLINK 1.5 release, Kafka0

Re: Ask for SQL using kafka in Flink

2018-06-02 Thread Radhya Sahal
Thanks Rong, I used Flink 1.3.0 in case of using Flink 1.5 how can I define jsonschema? Yes, there are two names but now I put one name only and I want to define jsonschema. Rong Rong wrote > Hi Radhya, > > Can you provide which Flink version you are using? Based on the latest > FLINK 1.5 rel

Re: Ask for SQL using kafka in Flink

2018-06-01 Thread Rong Rong
Hi Radhya, Can you provide which Flink version you are using? Based on the latest FLINK 1.5 release, Kafka09JsonTableSource takes: /** * Creates a Kafka 0.9 JSON {@link StreamTableSource}. * * @param topic Kafka topic to consume. * @param properties Properties for the Kafka consumer.

Ask for SQL using kafka in Flink

2018-06-01 Thread Radhya Sahal
Hi, Could anyone help me to solve this problem /Exception in thread "main" java.lang.Error: Unresolved compilation problem: The constructor Kafka09JsonTableSource(String, Properties, TypeInformation) is undefined / *--This is the code * public class FlinkKafkaSQL { public stat