Re: Flink 1.11 Table API cannot process Avro

2020-07-12 Thread Lian Jiang
Thanks Leonard and Jark. Here is my repo for your repro: https://bitbucket.org/jiangok/flink-playgrounds/src/0d242a51f02083711218d3810267117e6ce4260c/table-walkthrough/pom.xml#lines-131. As you can see, my pom.xml has already added flink-avro and avro dependencies. You can run this repro by: git

Re: Flink 1.11 Table API cannot process Avro

2020-07-12 Thread Leonard Xu
Hi, Jiang > Is there a uber jar or a list of runtime dependencies so that developers can > easily make the above example of Flink SQL for avro work? Thanks. The dependency list for using Avro in Flink SQL is simple and has not a uber jar AFAIK, we only need to add `flink-avro` and `avro` depend

Re: Flink 1.11 Table API cannot process Avro

2020-07-12 Thread Jark Wu
>From the latest exception message, it seems that the avro factory problem has been resolved. The new exception indicates that you don't have proper Apache Avro dependencies (because flink-avro doesn't bundle Apache Avro), so you have to add Apache Avro into your project dependency, or export HADOO

Re: Flink 1.11 Table API cannot process Avro

2020-07-12 Thread Lian Jiang
Thanks guys. I missed the runtime dependencies. After adding below into https://github.com/apache/flink-playgrounds/blob/master/table-walkthrough/Dockerfile. The original issue of "Could not find any factory for identifier" is gone. wget -P /opt/flink/lib/ https://repo1.maven.org/maven2/org/apach

Re: Flink 1.11 Table API cannot process Avro

2020-07-11 Thread Leonard Xu
Hi, Jiang > > jobmanager_1 | Available factory identifiers are: > jobmanager_1 | > jobmanager_1 | csv > jobmanager_1 | json > jobmanager_1 | parquet After added the flink-avro dependency, did you restart your cluster/sql-client? It looks flink-avro dependency did not l

Re: Flink 1.11 Table API cannot process Avro

2020-07-11 Thread 方盛凯
It seems that you don't add additional dependencies. org.apache.avro avro 1.8.2 Lian Jiang 于2020年7月12日周日 下午1:08写道: > i am using flink playground as the base: > > https://github.com/apache/flink-playgrounds/blob/master/table-walkthrough/pom.xml > > I observed "PhysicalLegacyTableSo

Re: Flink 1.11 Table API cannot process Avro

2020-07-11 Thread Lian Jiang
i am using flink playground as the base: https://github.com/apache/flink-playgrounds/blob/master/table-walkthrough/pom.xml I observed "PhysicalLegacyTableSourceScan". Not sure whether this is related. Thanks. Regards! On Sat, Jul 11, 2020 at 3:43 PM Lian Jiang wrote: > Thanks Jörn! > > I added

Re: Flink 1.11 Table API cannot process Avro

2020-07-11 Thread Lian Jiang
Thanks Jörn! I added the documented dependency in my pom.xml file: org.apache.flink flink-avro 1.11.0 The newly generated jar does have: $ jar tf target//spend-report-1.0.0.jar | grep FileSystemFormatFactory org/apache/flink/formats/parquet/ParquetFileSystemFormatFactory.class org/apache

Re: Flink 1.11 Table API cannot process Avro

2020-07-11 Thread Jörn Franke
You are missing additional dependencies https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/formats/avro.html > Am 11.07.2020 um 04:16 schrieb Lian Jiang : > >  > Hi, > > According to > https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/conne

Flink 1.11 Table API cannot process Avro

2020-07-10 Thread Lian Jiang
Hi, According to https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/filesystem.html, avro is supported for table API but below code failed: tEnv.executeSql("CREATE TABLE people (\n" + "id INT,\n" + "name STRING\n" + ") WITH (\n" +