Re: Discuss support for EXTERNAL/MANAGED semantics

2023-12-05 Thread Jack Ye
Yes I agree that ownership is the key concept here. But I think there is some value in connecting that to the external/managed table concept. *Re-define managed table concept under the multi-engine data lake world* I think the concept of EXTERNAL is not "not well-defined", but is just out of date.

Re: Discuss support for EXTERNAL/MANAGED semantics

2023-12-05 Thread Ryan Blue
Thanks for including a clear rationale for the proposal. I share Manu's concern about whether EXTERNAL is actually a well-defined concept. There are products that support external tables as read-only, but Spark is a good example of inconsistency. I think we've also had a suggestion in the past to a

Re: Discuss support for EXTERNAL/MANAGED semantics

2023-11-22 Thread Manu Zhang
Thanks Jack for initiating this valuable discussion. I'm also seeing issues with migrate procedure , where external / managed semantics need to be defined clearly. Nonetheless, is behavior consistent for *external* tables across engines and catalogs?

Discuss support for EXTERNAL/MANAGED semantics

2023-11-20 Thread Jack Ye
Hi everyone, As AWS Glue data catalog is now offering support for auto-compaction of Apache Iceberg tables , we have discovered some room in improving user experience for Iceberg use