Hi Sachin,
The increamental mongodb source is under package
`org.apache.flink.cdc.connectors.mongodb.source`, and the legacy source is
under package `org.apache.flink.cdc.connectors.mongodb`.
Here's an example:
-
imp
Thanks for the explanation.
One quick question how do I enable:
scan.incremental.snapshot.enabled = true
for my MongoDBSource ?
I don't see any option in the builder for the same.
Regards
Sachin
On Mon, Aug 19, 2024 at 8:00 PM Jiabao Sun wrote:
> Sorry, in my previous reply, I mistakenly w
Sorry, in my previous reply, I mistakenly wrote Flink 2.0 instead of Flink CDC
2.0.
I am correcting it here to avoid any misunderstanding.
Incremental snapshot reading is a new feature introduced in Flink CDC 2.0.
The following table shows the version mapping between Flink CDC Connectors and
Fl
Yes, it works.
Best,
Jiabo
On 2024/08/19 09:27:48 Sachin Mittal wrote:
> So basically if I set startupOptions(StartupOptions.initial())
> and also scan.incremental.snapshot.enabled = true
> Then it would read from the source in parallel, thereby reading the entire
> mongo collection faster.
>
>
So basically if I set startupOptions(StartupOptions.initial())
and also scan.incremental.snapshot.enabled = true
Then it would read from the source in parallel, thereby reading the entire
mongo collection faster.
Am I understanding that correctly?
Also I am using Flink 1.8, would it work with thi
Hi Sachin,
Incremental snapshot reading is a new feature introduced in Flink 2.0.
It has the following capabilities:
- Source can be parallel during snapshot reading to improve snapshot speed
- Source can perform checkpoints in the chunk granularity during snapshot
reading
Limitation:
- MongoD
Hi,
I am using mongodb cdc connector version 3.1.1
I am connecting to mongodb atlas, which uses mongodb version 7.0.
In the cdc connector I find a property:
scan.incremental.snapshot.enabled with default as false.
I wanted to know in what cases we should set this as true and what does
this prope
