Hi Tom, Thanks for volunteering here
I would suggest rewriting in a similar way as it was done for 1.20. We could rewrite Calcite dependency like it is done with others (for instance Janino) about Calcite As you mentioned there are multiple JIRA issues for Calcite upgrade somewhere in the past there was a suggestion to go version by version to reduce a bit changeset while upgrade Because of hard customization with rewriting some Calcite classes it is a separate store and not fast process (if compare with other dependency updates like json-path) Currently there are 2 PRs / JIRA issues to upgrade to 1.33.0[1] and 1.34.0[2] in waiting for review stage [1] https://issues.apache.org/jira/browse/FLINK-31362 [2] https://issues.apache.org/jira/browse/FLINK-31836 On Thu, Oct 31, 2024 at 3:37 PM Tom Cooper <c...@tomcooper.dev> wrote: > Hey Folks, > > I am new to Flink development and wanted to get my hands dirty by fixing > some CVEs. > > ## Flink Table > > I[raised this]( > https://apache-flink.slack.com/archives/C03GV7L3G2C/p1729865902686739) > previouslyon the #dev slack channel. Currently the flink-table-runtime is > using a version (2.7.0) of the json-path library which has a high severity > CVE ([CVE-2023-1370](https://nvd.nist.gov/vuln/detail/CVE-2023-1370)). We > have a JIRA task for this ([FLINK-33571]( > https://issues.apache.org/jira/browse/FLINK-33571)), the vulnerability is > fixed in 2.8.0 but that has an additional CVE, so 2.9.0 is the current > (clean) version. I have posted several PRs updating the lib to this new > version: > > - Flink 2.0: json-path is now managed via flink-shaded and so I posted a > [PR](https://github.com/apache/flink-shaded/pull/140) to add the 2.9.0 > version to the next flink-shaded release. So a further Flink PR will be > needed once Flink Shaded 0.20 is released. > > - Flink 1.20: In Flink 1.20 and earlier, json-path is managed directly in > the flink-table pom, so I posted a [PR]( > https://github.com/apache/flink/pull/25573) to update the lib on the > release-1.20 branch. It would be good to get this into a future 1.20.1 > release. > > ## Calcite > > I [raised this]( > https://apache-flink.slack.com/archives/C03GV7L3G2C/p1729867542212769) > previously on the #dev slack channel. The json-path CVEs also effect the > currently used Apache Calcite version (1.32). To fix that we would need to > update to Calcite 1.37+ ([FLINK-36602]( > https://issues.apache.org/jira/browse/FLINK-36602)). I have seen [many > issues]( > https://issues.apache.org/jira/browse/FLINK-35856?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20text%20~%20%22upgrade%20calcite%20version%22%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC) > in the upstream JIRA on Calcite updates and I realise it is not just a > simple version bump. > > I had a look at what would be involved and it seems we would need to > implement the now mandatory array parsing functionality, fix several > exception changes in the CalciteResource class and deal with changes to the > SqlParserTest that the flink parser tests inherit from. I am willing to > give this a try, I have some familiarity with Calcite, but I would like to > chat to someone more familiar with the SQL parser module and check for > hidden dragons. Also, would this update require a FLIP? > We could also just specifically override the json-path version in the > imported calcite 1.32, but I am not sure if that is a common / acceptable > practice in Flink. > > Any PR reviews/thoughts/help would be much appreciated. > > Tom Cooper > [@tomncooper](https://twitter.com/tomncooper) | [tomcooper.dev]( > https://tomcooper.dev/) -- Best regards, Sergey
