JingGe commented on code in PR #24788:
URL: https://github.com/apache/flink/pull/24788#discussion_r1611948679
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/utils/DynamicPartitionPruningUtils.java:
##########
@@ -234,20 +234,16 @@ private static boolean isSuitableFilter(RexNode
filterCondition) {
}
private void setTables(ContextResolvedTable catalogTable) {
- if (tables.size() == 0) {
- tables.add(catalogTable);
- } else {
- boolean hasAdded = false;
- for (ContextResolvedTable thisTable : new ArrayList<>(tables))
{
- if (hasAdded) {
- break;
- }
- if
(!thisTable.getIdentifier().equals(catalogTable.getIdentifier())) {
- tables.add(catalogTable);
- hasAdded = true;
- }
+ boolean alreadyExists = false;
+ for (ContextResolvedTable table : tables) {
Review Comment:
Sorry, I don't get your point. I meant looping the `Set` might have
performance issue.
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/utils/DynamicPartitionPruningUtils.java:
##########
@@ -234,20 +234,16 @@ private static boolean isSuitableFilter(RexNode
filterCondition) {
}
private void setTables(ContextResolvedTable catalogTable) {
- if (tables.size() == 0) {
- tables.add(catalogTable);
- } else {
- boolean hasAdded = false;
- for (ContextResolvedTable thisTable : new ArrayList<>(tables))
{
- if (hasAdded) {
- break;
- }
- if
(!thisTable.getIdentifier().equals(catalogTable.getIdentifier())) {
- tables.add(catalogTable);
- hasAdded = true;
- }
+ boolean alreadyExists = false;
+ for (ContextResolvedTable table : tables) {
+ if
(table.getIdentifier().equals(catalogTable.getIdentifier())) {
Review Comment:
This is a typical hash map logic
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/utils/DynamicPartitionPruningUtils.java:
##########
@@ -234,20 +234,16 @@ private static boolean isSuitableFilter(RexNode
filterCondition) {
}
private void setTables(ContextResolvedTable catalogTable) {
- if (tables.size() == 0) {
- tables.add(catalogTable);
- } else {
- boolean hasAdded = false;
- for (ContextResolvedTable thisTable : new ArrayList<>(tables))
{
- if (hasAdded) {
- break;
- }
- if
(!thisTable.getIdentifier().equals(catalogTable.getIdentifier())) {
- tables.add(catalogTable);
- hasAdded = true;
- }
+ boolean alreadyExists = false;
+ for (ContextResolvedTable table : tables) {
Review Comment:
I think there are many reasons to use `Map` instead of `Set`:
1. the logic is point search instead of loop search as I mentioned below.
2. O(1) than O(n) for better performance, because the The
`DynamicPartitionPruningUtils` class will be used centrally for batch jobs[1],
i.e. for large projects with many tables, it could be a bottleneck.
3. less code while using e.g. Map.putIfAbsent​(K, V)
[1]
https://github.com/apache/flink/blob/0737220959fe52ee22535e7db55b015a46a6294e/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/optimize/program/FlinkDynamicPartitionPruningProgram.java#L103
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
