Re: can not use iceberg as a sql source in flink sql according to iceberg 0.12.0

2021-09-27 Thread Ryan Blue
Thanks, Josh! I guess it makes sense that without the base class you can't load the Parquet class. We'll have to watch out for Hadoop/Flink issues. I think we hit one as well where not having Configuration in the Flink classpath could prevent loading modules correctly. Ryan On Sun, Sep 26, 2021

Re: can not use iceberg as a sql source in flink sql according to iceberg 0.12.0

2021-09-26 Thread Joshua Fan
Hi I found the reason why this exception 'java.lang.NoClassDefFoundError: org/apache/iceberg/shaded/org/apache/parquet/hadoop/ParquetInputFormat' was raised. Actually, it was because of the absence of class 'org/apache/hadoop/mapreduce/lib/input/FileInputFormat'. After I put the hadoop-mapreduce-

Re: can not use iceberg as a sql source in flink sql according to iceberg 0.12.0

2021-09-26 Thread Joshua Fan
Hi openinx I do not get you. what do you mean by 'Looks like the line 112 in HadoopReadOptions is not the first line accessing the variables in ParquetInputFormat.'? The parquet file I want to read was wrote by iceberg table without any explicit specified, no file format and no parquet version was

Re: can not use iceberg as a sql source in flink sql according to iceberg 0.12.0

2021-09-22 Thread OpenInx
Hi Joshua Can you check what's the parquet version you are using ? Looks like the line 112 in HadoopReadOptions is not the first line accessing the variables in ParquetInputFormat. [image: image.png] On Wed, Sep 22, 2021 at 11:07 PM Joshua Fan wrote: > Hi > I am glad to use iceberg as table