Re: Questions about using Flink MongoDB CDC

2024-11-17 Thread Jiabao Sun
Hi, The Flink SQL Planner uses the ChangelogNormalize operator to cache all incoming data for upsert type Changelog in order to complete the pre-image values, which results in additional state overhead. When the MongoDB version is below 6.0, the oplog does not contain Pre-Images of changed record

Questions about using Flink MongoDB CDC

2024-11-16 Thread wangye...@yeah.net
Hi all: While using Flink with MongoDB CDC, I've noticed that my Flink job causes MongoDB's memory usage to continuously increase. Below, I will detail the specific scenario to help identify where the issue lies. 1. MongoDB deployment architecture: sharded. 2. The memory usage of the