Hi, There was a proposal along the same lines, for the read portion few weeks back by Ashvin.
https://docs.google.com/document/d/1yzLXSOtzBXyaWHfeVsWsMu4xmOH8rV6QyM5ZAnJZjMQ/edit?usp=drivesdk Thanks, Vignesh. On Tue, Nov 26, 2024, 11:59 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi Nikhil > > Thanks for your message, very interesting. > > I think it would be great to involve the Polaris project here as well, > as a REST Catalog implementation. > The Polaris community is discussing storage/backend right now, so it > would be the perfect timing to consider leveraging S3 conditional > writes (as a plugin for instance first). > > I would be happy to connect and know more about your perspective about > that. > > Thanks, > Regards > JB > > PS: I will be at AWS re:Invent next week, so maybe we can connect there. > > On Tue, Nov 26, 2024 at 6:35 PM Nikhil Benesch <nikhil.bene...@gmail.com> > wrote: > > > > Hi all, > > > > With Amazon S3 announcing support for the If-Match header yesterday [0], > all the > > major object store implementations now support a compare-and-swap > operation. > > > > As far as I can tell, this opens up the possibility of storing Iceberg > > catalogs directly on object storage, without the need for a separate > metastore, > > and without violating any of Iceberg's ACID guarantees. > > > > It seems the immediate next step is to build an independent Java or REST > catalog > > backend to prove this concept out. Long term, though, the ideal would be > to > > have such a catalog backend be a first class citizen in the Iceberg > project. > > > > Is anyone else in the Iceberg community barking up this tree? I'm a long > term > > Iceberg enthusiast, but new to the community. I'd very much appreciate > any > > pointers to current or past discussions on the topic. So far all I've > been > > able to turn up is some light chatter from myself and others on Bluesky > and > > Hacker News ([1][2][3]). > > > > Cheers, > > Nikhil > > > > [0]: > https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-s3-functionality-conditional-writes/ > > [1]: https://bsky.app/profile/benesch.bsky.social/post/3lauesxg3ic2c > > [2]: https://bsky.app/profile/eatonphil.bsky.social/post/3lbskq3jwk22e > > [3]: https://news.ycombinator.com/item?id=42240370 >
