Re: Welcome new PMC members!

2021-11-19 Thread hilili...@gmail.com
congratulations! https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=3&name=liwei+li&uid=hililiwei%40gmail.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fwzpmmc%2F8a024ff09112e3c95cf74831f92b8360.jpg&items=%5B%22hililiwei%40gmail.com%22%5D"; width="400" styl

Re: Help reading iceberg data from databricks

2021-11-19 Thread Jack Ye
I am not familiar with this environment setup, but I heard that because Iceberg requires Spark extensions and Databricks has its own extensions loaded automatically it causes conflicts and unexpected behaviors. Could you provide more details for why the cluster stops? Is there any stacktrace you c

Re: Help reading iceberg data from databricks

2021-11-19 Thread Ryan Blue
Hi Ridam, Do you have any additional information that would be helpful? When you say your cluster stops, what happens? Do you get an error message? What metadata files are you loading and what did you run? Thanks, Ryan On Fri, Nov 19, 2021 at 2:06 PM Ridam Arora wrote: > Hi everyone, > > I'm t

Help reading iceberg data from databricks

2021-11-19 Thread Ridam Arora
Hi everyone, I'm trying to query data on databricks using iceberg tables saved on S3, but not able to figure out , my cluster stops everytime I load metadata files into it even though the amount of data is very small. Is there any documentation related to it? Any help would be appreciated. Thanks

Re: Welcome new PMC members!

2021-11-19 Thread Armin Najafi
Congrats Jack and Russell! πŸ˜€πŸŽ‰ Cheers, Armin On Thu, Nov 18, 2021 at 8:37 PM Rajarshi Sarkar wrote: > Congrats, Jack and Russell !! > > Well deserved !! > > Regards, > Raj > > > On Fri, Nov 19, 2021 at 7:02 AM xingbo wu wrote: > >> Congrats, Jack and Russell! Well deserved! >> >> Yan Yan 于2021

Re: Regarding the support of pluggable procedures in Iceberg

2021-11-19 Thread Ryan Blue
Thanks for the additional context. #1 makes more sense now. What are you thinking about exposing that isn't in the standard set of procedures? Are there procedures that you'd want for an Iceberg table when using Nessie as the catalog? Would it work to add a SupportsProcedures interface on the Icebe

Re: Regarding the support of pluggable procedures in Iceberg

2021-11-19 Thread Ryan Murray
Hey Ryan, Thanks for the follow up. As I see it the use cases are as follows: 1) add more iceberg specific procedures to the existing catalog. This is when we are actually operating on iceberg tables (not "to expose custom stored procedures for non-Iceberg systems") 2) modify existing OSS iceberg