Re: Re: Guidance on implementing Hybrid CDC Pattern from "CDC patterns in Apache Iceberg" talk

2024-01-04 Thread Nick Del Nano
er ways of solving this problem? Is there a simple or clever way to implement this for SQL clients? If I can sort this out for SQL clients, I'd feel closer to getting the best of all worlds with this approach. Cheers, Nick On Mon, Sep 25, 2023 at 12:21 PM Nick Del Nano wrote: > Thanks ev

RE: Re: Guidance on implementing Hybrid CDC Pattern from "CDC patterns in Apache Iceberg" talk

2023-09-25 Thread Nick Del Nano
gt; https://iceberg.apache.org/docs/latest/branching/. So essentially, run > >> the MERGE and then tag it with a known name that users can then use for > >> reading data as of a snapshot that has CDC data merged as opposed to always > >> reading the latest snapshot. > &g

Guidance on implementing Hybrid CDC Pattern from "CDC patterns in Apache Iceberg" talk

2023-09-22 Thread Nick Del Nano
Hi, I am exploring implementing the Hybrid CDC Pattern explained at 29:26 in Ryan Blue's talk CDC patterns in Apache Iceberg . The use case is: 1. Stream CDC logs to