Hi Nikhil, I am also personally very excited about S3 adding this support!
I would suggest we discuss this after the AWS re:invent 2024 event that is coming right next week, as there are going to be more S3 feature announcements during that week, and the community can have a more comprehensive discussion after knowing all the new S3 features that might help this domain :) Best, Jack Ye On Tue, Nov 26, 2024 at 9:36 AM Nikhil Benesch <nikhil.bene...@gmail.com> wrote: > Hi all, > > With Amazon S3 announcing support for the If-Match header yesterday [0], > all the > major object store implementations now support a compare-and-swap > operation. > > As far as I can tell, this opens up the possibility of storing Iceberg > catalogs directly on object storage, without the need for a separate > metastore, > and without violating any of Iceberg's ACID guarantees. > > It seems the immediate next step is to build an independent Java or REST > catalog > backend to prove this concept out. Long term, though, the ideal would be to > have such a catalog backend be a first class citizen in the Iceberg > project. > > Is anyone else in the Iceberg community barking up this tree? I'm a long > term > Iceberg enthusiast, but new to the community. I'd very much appreciate any > pointers to current or past discussions on the topic. So far all I've been > able to turn up is some light chatter from myself and others on Bluesky and > Hacker News ([1][2][3]). > > Cheers, > Nikhil > > [0]: > https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-s3-functionality-conditional-writes/ > [1]: https://bsky.app/profile/benesch.bsky.social/post/3lauesxg3ic2c > [2]: https://bsky.app/profile/eatonphil.bsky.social/post/3lbskq3jwk22e > [3]: https://news.ycombinator.com/item?id=42240370 >
