Re: Help reading iceberg data from databricks

2021-11-19 Thread Jack Ye
I am not familiar with this environment setup, but I heard that because Iceberg requires Spark extensions and Databricks has its own extensions loaded automatically it causes conflicts and unexpected behaviors. Could you provide more details for why the cluster stops? Is there any stacktrace you c

Re: Help reading iceberg data from databricks

2021-11-19 Thread Ryan Blue
Hi Ridam, Do you have any additional information that would be helpful? When you say your cluster stops, what happens? Do you get an error message? What metadata files are you loading and what did you run? Thanks, Ryan On Fri, Nov 19, 2021 at 2:06 PM Ridam Arora wrote: > Hi everyone, > > I'm t

Help reading iceberg data from databricks

2021-11-19 Thread Ridam Arora
Hi everyone, I'm trying to query data on databricks using iceberg tables saved on S3, but not able to figure out , my cluster stops everytime I load metadata files into it even though the amount of data is very small. Is there any documentation related to it? Any help would be appreciated. Thanks