In regards to the multiple emails sent earlier, please use this one for discussions.
Thanks you! On 2023/12/07 00:47:42 Drew wrote: > Hi everyone, > > My name is Drew Gallardo, and I’m a part of the Iceberg team at Amazon EMR > and Athena. I’m reaching out to share a proposal that introduces data > commits as a part of the RESTCatalog. The current process for data commits > lives on the client side, and by shifting this logic into the REST catalog, > we can empower the catalog service with more control of this process. > > This proposal addresses specific use cases that showcase the benefits of > moving the commit logic to the service side. For instance, this shift > allows the user to refine conflict resolution mechanisms, giving precedence > to operations that modify the table state to ensure their completion > without conflict. Furthermore, our POC demonstrated an improvement in the > success rate of concurrent write operations against the GlueCatalog. This > all can be found in the detailed proposal below. Feel free to comment, and > add your suggestions! > > Detailed proposal: > https://docs.google.com/document/d/1OG68EtPxLWvNBJACQwcMrRYuGJCnQas8_LSruTRcHG8/edit?usp=sharing > Github POC: https://github.com/apache/iceberg/pull/9237 > > Looking forward to hearing back > > Thanks, > > Drew Gallardo > Amazon EMR & Athena > d...@amazon.com >
