Ah, fascinating. Thanks very much for the pointer. Here's the thread introducing the proposal [0], for anyone else curious.
[0]: https://lists.apache.org/thread/kh4n98w4z22sc8h2vot4q8n44vdtnltg On Tue, Nov 26, 2024 at 3:27 PM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > > Hi Vignesh > > Thanks for the reminder, I remember we quickly discussed this during a > community meeting. > > I will take a new look at the doc. > > Regards > JB > > On Tue, Nov 26, 2024 at 9:19 PM Vignesh <vignesh.v...@gmail.com> wrote: > > > > Hi, > > There was a proposal along the same lines, for the read portion few weeks > > back by Ashvin. > > > > https://docs.google.com/document/d/1yzLXSOtzBXyaWHfeVsWsMu4xmOH8rV6QyM5ZAnJZjMQ/edit?usp=drivesdk > > > > Thanks, > > Vignesh. > > > > > > On Tue, Nov 26, 2024, 11:59 AM Jean-Baptiste Onofré <j...@nanthrax.net> > > wrote: > >> > >> Hi Nikhil > >> > >> Thanks for your message, very interesting. > >> > >> I think it would be great to involve the Polaris project here as well, > >> as a REST Catalog implementation. > >> The Polaris community is discussing storage/backend right now, so it > >> would be the perfect timing to consider leveraging S3 conditional > >> writes (as a plugin for instance first). > >> > >> I would be happy to connect and know more about your perspective about > >> that. > >> > >> Thanks, > >> Regards > >> JB > >> > >> PS: I will be at AWS re:Invent next week, so maybe we can connect there. > >> > >> On Tue, Nov 26, 2024 at 6:35 PM Nikhil Benesch <nikhil.bene...@gmail.com> > >> wrote: > >> > > >> > Hi all, > >> > > >> > With Amazon S3 announcing support for the If-Match header yesterday [0], > >> > all the > >> > major object store implementations now support a compare-and-swap > >> > operation. > >> > > >> > As far as I can tell, this opens up the possibility of storing Iceberg > >> > catalogs directly on object storage, without the need for a separate > >> > metastore, > >> > and without violating any of Iceberg's ACID guarantees. > >> > > >> > It seems the immediate next step is to build an independent Java or REST > >> > catalog > >> > backend to prove this concept out. Long term, though, the ideal would be > >> > to > >> > have such a catalog backend be a first class citizen in the Iceberg > >> > project. > >> > > >> > Is anyone else in the Iceberg community barking up this tree? I'm a long > >> > term > >> > Iceberg enthusiast, but new to the community. I'd very much appreciate > >> > any > >> > pointers to current or past discussions on the topic. So far all I've > >> > been > >> > able to turn up is some light chatter from myself and others on Bluesky > >> > and > >> > Hacker News ([1][2][3]). > >> > > >> > Cheers, > >> > Nikhil > >> > > >> > [0]: > >> > https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-s3-functionality-conditional-writes/ > >> > [1]: https://bsky.app/profile/benesch.bsky.social/post/3lauesxg3ic2c > >> > [2]: https://bsky.app/profile/eatonphil.bsky.social/post/3lbskq3jwk22e > >> > [3]: https://news.ycombinator.com/item?id=42240370
