snuyanzin opened a new pull request, #27250:
URL: https://github.com/apache/flink/pull/27250
## What is the purpose of the change
The current issue is that Changelog normalize uses Calcite's CNF in order to
calculate common conditions between several changelog normalize nodes if they
share same table scan. In case of query with huge amount of conditions
connected with `OR` it might lead to OOM even if another changelog normalize
nodes doesn't have conditions.
This code removes usage of CNF in marking program and fixes the behavior for
the case of other changelog normalize nodes have no conditions.
The rest part of the issue will be solved separately
## Brief change log
FlinkMarkChangelogNormalizeProgram.java
## Verifying this change
1. existing tests
2. I have locally query leading to OOM, not sure if a huge query (about 1k
lines including table/view creation) should be a part of test...
3. just check that there is no occurrences of `toCnf` in that class anymore
and existing tests continue passing
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): ( no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
