This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a change to branch 2.1_47093 in repository https://gitbox.apache.org/repos/asf/doris.git
discard 6f636682949 branch-2.1: [fix](Nereids) let anonymous alias same as user input #47093 add 2f7e0f0b761 Revert "[branch-2.1]downgrade resource tag when there is not queryabl… (#47746) add c8ccada010a branch-2.1: [Fix](memory) Add try catch block for `Segment::load_pk_index_and_bf`… (#47742) add 6a3c630ee0c branch-2.1: [chore](binlog) Save properties in CreateTableRecord #47551 (#47639) add 964c9401312 [fix](coordinator) Fix wrong `recvrId` in fragment contains BHJ (#47727) (#47729) add e2f9b436a4b [fix](nereids)Add catalog/db/table filter info in SchemaScanNode (#46864) (#47759) add 7b579c75c6a [fix](ES Catalog)Make sure ES meta is synced before using (#46781) (#47711) add a121566ce1c branch-2.1: [fix](Nereids) fix log bug and add more numeric arithmetic function cases #47228 (#47749) add 5fade758912 [test](mtmv)Pick some test pr to 21 #46299 #46787 #46546 (#47098) add 402570f603e [fix](case) disable testSelect in StmtExecutorTest (#47760) add 3ee7ad9b817 branch-2.1: [fix](test) fix test_routine_load case unstable #47601 (#47752) add b03d9663986 [fix](nereids)use visitNullableAggregateFunction for NullableAggregateFunction (#47716) add b4f03c0eb3b [charry-pick](branch-2.1) add more hudi jni test cases #47431 (#47764) add b4af6713478 branch-2.1: [Fix](catalog)Fixes query failures for Paimon tables stored in Kerberized HDFS #47192 (#47694) add de27cbb9ce6 branch-2.1: [Chore](dependencies)upgrade dependencies #47127 (#47765) add a3e93fa3b65 branch-2.1: [Fix](job)Fix for Duplicate Scheduling of Tasks #46872 (#46899) add da3222e7e16 branch-2.1: [fix](external catalog) Fix missing fields when rebuilding metadata from image (#47751) add 3ca0fc3784a branch-2.1: [fix](external catalog) Persisting the External Catalog comment field… (#47569) add 217e6cef340 [Bug](join) return eof when join build sink awakend by downstream source #47380 (#47709) add 818e675b728 branch-2.1: [fix](restore) Cut down restore timeout when create replicas failed #47278 (#47496) add f55fba7ceaf branch-2.1: [debug](move-memtable) add debug logs for skip_two_backends injection cases #46826 (#46945) add 3f1a9db94fb branch-2.1: [fix](Nereids) Use the schema saved during planning as the schema of the original target table #47337 (#47773) add e8531621c00 branch-2.1: [feat](binlog) Support getting binlogs in batch #47557 (#47640) add cb3e77379dd branch-2.1: [fix](backup) filter the staled task response by job id #47416 (#47445) add f85176bee04 branch-2.1: [improve](move-memtable) improve error log and message for "not enough streams" #47470 (#47722) add eee254efab5 [regression-test](case) remove test_auto_partition_behavior before fix (#47818) add 48bcdf539a0 branch-2.1: [fix](table) Supply rollup for get ddl for sync #47732 (#47796) add 8c6c6424626 branch-2.1: [fix](nereids)keep at least one hash output slot when prune slots in hash join node #47318 (#47683) add f8fb5f90b81 branch-2.1: [fix](Nereids) not release lock if schema changed when insert #47733 (#47833) add c4a10849d54 branch-2.1: [fix](nereids) fix nereids dead loop due to simplify range output in-predicate's options in random order #47830 (#47828) add 058929528c4 branch-2.1: [fix](mv) Fix use sync mv wrongly when use rbo materialized view rewrite rule #47650 (#47835) add 13c8b1797db [fix](nereids)fix logical repeat computefd (#47798) add 8fb10b0fa02 branch-2.1: [opt](paimon) Optimize error prompt #47593 (#47790) add 3ec900e8784 branch-2.1:[opt](jdbc catalog) Compatible with higher ClickHouse JDBC Driver versions (#47832) add 3e454445ca0 branch-2.1: [fix](Nereids) let anonymous alias same as user input #47093 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (6f636682949) \ N -- N -- N refs/heads/2.1_47093 (3e454445ca0) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../exec/schema_scanner/schema_tables_scanner.cpp | 3 + be/src/exprs/runtime_filter_slots.h | 3 + be/src/olap/rowset/segment_v2/segment.cpp | 10 +- be/src/pipeline/exec/hashjoin_build_sink.cpp | 11 +- .../pipeline_x/local_exchange/local_exchanger.cpp | 14 + be/src/vec/runtime/shared_hash_table_controller.h | 1 - be/src/vec/sink/writer/vtablet_writer_v2.cpp | 31 +- fe/be-java-extensions/hadoop-hudi-scanner/pom.xml | 1 - .../apache/doris/hudi/HadoopHudiJniScanner.java | 55 +- fe/be-java-extensions/paimon-scanner/pom.xml | 6 - .../org/apache/doris/paimon/PaimonJniScanner.java | 30 +- .../authentication/AuthenticationConfig.java | 17 +- .../PreExecutionAuthenticatorCache.java | 87 +++ .../org/apache/doris/backup/BackupHandler.java | 28 +- .../java/org/apache/doris/backup/RestoreJob.java | 13 +- .../org/apache/doris/binlog/BinlogManager.java | 11 +- .../java/org/apache/doris/binlog/BinlogUtils.java | 17 +- .../org/apache/doris/binlog/CreateTableRecord.java | 10 + .../java/org/apache/doris/binlog/DBBinlog.java | 6 +- .../java/org/apache/doris/binlog/TableBinlog.java | 4 +- .../main/java/org/apache/doris/catalog/Env.java | 30 + .../java/org/apache/doris/catalog/EsTable.java | 15 +- .../apache/doris/common/MarkedCountDownLatch.java | 6 + .../apache/doris/datasource/ExternalCatalog.java | 27 +- .../apache/doris/datasource/ExternalDatabase.java | 36 +- .../doris/datasource/FederationBackendPolicy.java | 3 - .../apache/doris/datasource/InitCatalogLog.java | 7 +- .../apache/doris/datasource/InitDatabaseLog.java | 7 +- .../doris/datasource/jdbc/JdbcExternalCatalog.java | 2 +- .../jdbc/client/JdbcClickHouseClient.java | 130 ++++ .../doris/datasource/jdbc/client/JdbcClient.java | 12 + .../jdbc/client/JdbcOceanBaseClient.java | 2 +- .../datasource/paimon/PaimonExternalCatalog.java | 16 +- .../org/apache/doris/httpv2/rest/LoadAction.java | 2 - .../doris/job/base/JobExecutionConfiguration.java | 2 +- .../java/org/apache/doris/mysql/MysqlProto.java | 3 +- .../org/apache/doris/mysql/privilege/Auth.java | 9 - .../mysql/privilege/CommonUserProperties.java | 11 - .../apache/doris/mysql/privilege/UserProperty.java | 22 - .../doris/mysql/privilege/UserPropertyMgr.java | 8 - .../org/apache/doris/nereids/StatementContext.java | 7 + .../glue/translator/PhysicalPlanTranslator.java | 20 +- .../doris/nereids/jobs/executor/Rewriter.java | 4 +- .../generator/PlanPatternGeneratorAnalyzer.java | 4 +- .../org/apache/doris/nereids/rules/RuleType.java | 1 + .../nereids/rules/analysis/CollectRelation.java | 14 +- .../rules/expression/rules/SimplifyRange.java | 55 +- .../LogicalSchemaScanToPhysicalSchemaScan.java | 5 +- .../rewrite/PushDownFilterIntoSchemaScan.java | 77 +++ .../mv/SelectMaterializedIndexWithAggregate.java | 16 +- .../functions/executable/NumericArithmetic.java | 6 +- .../visitor/AggregateFunctionVisitor.java | 4 +- .../commands/insert/InsertIntoTableCommand.java | 18 +- .../trees/plans/commands/insert/InsertUtils.java | 9 +- .../nereids/trees/plans/logical/LogicalRepeat.java | 9 +- .../trees/plans/logical/LogicalSchemaScan.java | 72 ++- .../trees/plans/physical/PhysicalSchemaScan.java | 60 +- .../planner/BackendPartitionedSchemaScanNode.java | 5 +- .../org/apache/doris/planner/OlapScanNode.java | 50 +- .../org/apache/doris/planner/SchemaScanNode.java | 6 +- .../apache/doris/planner/SingleNodePlanner.java | 5 +- .../java/org/apache/doris/qe/ConnectContext.java | 8 +- .../java/org/apache/doris/qe/ConnectProcessor.java | 3 +- .../main/java/org/apache/doris/qe/Coordinator.java | 2 - .../java/org/apache/doris/qe/SessionVariable.java | 10 +- .../apache/doris/service/FrontendServiceImpl.java | 23 +- .../org/apache/doris/system/BeSelectionPolicy.java | 34 +- .../apache/doris/catalog/CreateTableLikeTest.java | 38 ++ .../doris/datasource/ExternalCatalogTest.java | 34 + .../jdbc/client/JdbcClickHouseClientTest.java | 67 ++ .../jdbc/{ => client}/JdbcClientExceptionTest.java | 4 +- .../paimon/PaimonExternalCatalogTest.java} | 31 +- .../job/base/JobExecutionConfigurationTest.java | 59 +- .../properties/FunctionalDependenciesTest.java | 15 +- .../apache/doris/planner/ResourceTagQueryTest.java | 12 +- .../java/org/apache/doris/qe/StmtExecutorTest.java | 4 +- .../apache/doris/system/SystemInfoServiceTest.java | 23 - fe/pom.xml | 10 +- gensrc/thrift/FrontendService.thrift | 2 + regression-test/conf/regression-conf.groovy | 7 - .../test_column_prune_in_hash_join.out | Bin 0 -> 255 bytes .../jdbc/test_clickhouse_jdbc_catalog.out | Bin 4381 -> 8056 bytes .../hudi/test_hudi_incremental.out | Bin 5687 -> 7551 bytes .../hudi/test_hudi_schema_evolution.out | Bin 7521 -> 11234 bytes .../external_table_p2/hudi/test_hudi_snapshot.out | Bin 696105 -> 1044838 bytes .../hudi/test_hudi_timetravel.out | Bin 2111 -> 3119 bytes .../test_information_schema.out | Bin 0 -> 859 bytes .../mv_p0/agg_have_dup_base/agg_have_dup_base.out | Bin 297 -> 306 bytes .../agg_use_key_direct/agg_use_key_direct.out} | Bin 165 -> 210 bytes .../{no_await/no_await.out => await/await.out} | Bin 3495 -> 3493 bytes regression-test/data/mv_p0/no_await/no_await.out | Bin 3495 -> 3493 bytes .../mv/external_table/part_partition_invalid.out | Bin 2031 -> 1324 bytes .../nereids_syntax_p0/mv/newMv/multi_slot4.out | Bin 188 -> 264 bytes .../workload_manager_p0/skip_rg_test_table.csv | 2 - .../org/apache/doris/regression/suite/Suite.groovy | 210 +++++-- .../pipeline/p0/conf/regression-conf.groovy | 1 + .../suites/auth_p0/test_select_column_auth.groovy | 2 +- .../test_column_prune_in_hash_join.groovy | 131 ++++ .../suites/correctness_p0/test_mv_case.groovy | 13 +- .../create_view_nereids/create_view_use_mv.groovy | 4 +- .../external_table_p0/es/test_es_query.groovy | 30 +- .../es/test_es_query_no_http_url.groovy | 31 +- .../jdbc/test_clickhouse_jdbc_catalog.groovy | 73 ++- .../es/test_external_catalog_es.groovy | 52 -- .../external_table_p2/es/test_external_es.groovy | 56 -- .../hudi/test_hudi_incremental.groovy | 17 +- .../hudi/test_hudi_schema_evolution.groovy | 11 +- .../hudi/test_hudi_snapshot.groovy | 15 +- .../hudi/test_hudi_timetravel.groovy | 15 +- .../test_multi_replica_fault_injection.groovy | 2 +- .../test_information_schema.groovy | 69 ++ .../routine_load/ddl/agg_tbl_basic_drop.sql | 2 +- .../routine_load/ddl/dup_tbl_array_drop.sql | 2 +- .../routine_load/ddl/dup_tbl_basic_drop.sql | 2 +- .../ddl/dup_tbl_basic_multi_table_drop.sql | 2 +- .../routine_load/ddl/mow_tbl_array_drop.sql | 2 +- .../routine_load/ddl/mow_tbl_basic_drop.sql | 2 +- .../routine_load/ddl/uniq_tbl_array_drop.sql | 2 +- .../routine_load/ddl/uniq_tbl_basic_drop.sql | 2 +- .../ddl/uniq_tbl_basic_drop_sequence.sql | 2 +- .../agg_have_dup_base/agg_have_dup_base.groovy | 6 +- .../mv_p0/agg_state/test_agg_state_max_by.groovy | 2 +- .../agg_use_key_direct/agg_use_key_direct.groovy | 63 ++ .../no_await.groovy => await/await.groovy} | 61 +- .../suites/mv_p0/case_ignore/case_ignore.groovy | 1 + .../suites/mv_p0/count_star/count_star.groovy | 2 +- .../suites/mv_p0/dis_26495/dis_26495.groovy | 2 + .../suites/mv_p0/k1ap2spa/k1ap2spa.groovy | 4 +- .../mv_p0/k1s2m3_auto_inc/k1s2m3_auto_inc.groovy | 2 +- .../multi_agg_with_same_slot.groovy | 2 +- .../mv_p0/multi_slot_k123p/multi_slot_k123p.groovy | 4 +- .../multi_slot_k1a2p2ap3p.groovy | 2 +- .../multi_slot_k1a2p2ap3ps.groovy | 3 +- .../multi_slot_k1p2ap3p/multi_slot_k1p2ap3p.groovy | 2 +- .../multi_slot_k1p2ap3ps.groovy | 2 +- .../multi_slot_multi_mv/multi_slot_multi_mv.groovy | 4 + .../suites/mv_p0/mv_with_view/mv_with_view.groovy | 1 + .../suites/mv_p0/no_await/no_await.groovy | 21 + .../mv_p0/routine_load_hll/routine_load_hll.groovy | 2 +- .../mv_p0/ssb/multiple_ssb/multiple_ssb.groovy | 1 + .../suites/mv_p0/ssb/q_4_1_r1/q_4_1_r1.groovy | 2 +- .../mv_p0/sum_divede_count/sum_devide_count.groovy | 2 +- .../suites/mv_p0/test_28741/test_28741.groovy | 2 +- .../test_approx_count_distinct.groovy | 2 +- .../suites/mv_p0/test_base/test_base.groovy | 2 +- .../mv_p0/test_casewhen/test_casewhen.groovy | 2 +- .../suites/mv_p0/test_doc_e4/test_doc_e4.groovy | 2 +- .../test_dup_group_by_mv_abs.groovy | 2 +- .../test_dup_group_by_mv_plus.groovy | 2 +- .../mv_p0/test_dup_mv_abs/test_dup_mv_abs.groovy | 2 +- .../mv_p0/test_dup_mv_bin/test_dup_mv_bin.groovy | 3 +- .../test_dup_mv_bitmap_hash.groovy | 3 +- .../mv_p0/test_dup_mv_plus/test_dup_mv_plus.groovy | 2 +- .../mv_p0/test_dup_mv_year/test_dup_mv_year.groovy | 3 +- .../suites/mv_p0/test_ndv/test_ndv.groovy | 2 +- .../suites/mv_p0/test_nvl/test_nvl.groovy | 1 + .../suites/mv_p0/test_o2/test_o2.groovy | 2 +- .../suites/mv_p0/test_substr/test_substr.groovy | 2 +- .../mv_p0/test_tbl_name/test_tbl_name.groovy | 2 +- .../mv_p0/test_upper_alias/test_upper_alias.groovy | 3 +- .../testAggQueryOnAggMV10.groovy | 2 +- .../testAggQueryOnAggMV11.groovy | 3 +- .../testAggQueryOnAggMV2.groovy | 2 +- .../testAggQueryOnAggMV3.groovy | 2 +- .../testAggQuqeryOnAggMV5.groovy | 3 +- .../testAggQuqeryOnAggMV6.groovy | 3 +- .../testAggQuqeryOnAggMV7.groovy | 5 +- .../testAggTableCountDistinctInBitmapType.groovy | 3 + .../testAggregateMVCalcAggFunctionQuery.groovy | 2 +- .../testBitmapUnionInQuery.groovy | 3 +- .../testCountDistinctToBitmap.groovy | 1 + .../testIncorrectMVRewriteInSubquery.groovy | 3 +- .../testIncorrectRewriteCountDistinct.groovy | 3 +- .../testJoinOnLeftProjectToJoin.groovy | 3 +- .../mv_p0/ut/testNDVToHll/testNDVToHll.groovy | 3 +- .../testOrderByQueryOnProjectView.groovy | 3 +- .../ut/testProjectionMV1/testProjectionMV1.groovy | 3 +- .../ut/testProjectionMV2/testProjectionMV2.groovy | 3 +- .../ut/testProjectionMV3/testProjectionMV3.groovy | 3 +- .../ut/testProjectionMV4/testProjectionMV4.groovy | 3 +- .../testSingleMVMultiUsage.groovy | 2 +- .../mv_p0/ut/testSubQuery/testSubQuery.groovy | 3 +- .../fold_constant_numeric_arithmatic.groovy | 696 +++++++++++---------- .../suites/nereids_p0/hint/test_use_mv.groovy | 4 +- .../expression/test_simplify_range.groovy | 34 + .../mv_contain_external_table.groovy | 3 +- .../external_table/part_partition_invalid.groovy | 31 +- .../mv/external_table/single_external_table.groovy | 6 +- .../mv/is_in_debug_mode/is_in_debug_mode.groovy | 3 + .../mv/nested_mtmv/nested_mtmv.groovy | 6 +- .../mv/union_rewrite/usercase_union_rewrite.groovy | 2 +- .../nereids_syntax_p0/mv/newMv/case_ignore.groovy | 3 +- .../mv/newMv/dup_gb_mv_abs.groovy | 3 +- .../mv/newMv/dup_gb_mv_plus.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/dup_mv_abs.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/dup_mv_bin.groovy | 3 +- .../mv/newMv/dup_mv_bm_hash.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/dup_mv_plus.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/dup_mv_year.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/multi_slot1.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/multi_slot2.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/multi_slot3.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/multi_slot4.groovy | 11 +- .../nereids_syntax_p0/mv/newMv/multi_slot5.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/multi_slot6.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/mv_with_view.groovy | 2 + .../nereids_syntax_p0/mv/newMv/single_slot.groovy | 3 +- .../mv/newMv/sum_devide_count.groovy | 3 +- .../nereids_syntax_p0/mv/newMv/unique_mv.groovy | 3 +- .../nereids_syntax_p0/mv/ut/MVMultiUsage.groovy | 3 +- .../suites/nereids_syntax_p0/mv/ut/MVWithAs.groovy | 4 +- .../nereids_syntax_p0/mv/ut/aggCDInBitmap.groovy | 2 +- .../nereids_syntax_p0/mv/ut/aggMVCalcAggFun.groovy | 3 +- .../nereids_syntax_p0/mv/ut/aggOnAggMV1.groovy | 3 +- .../nereids_syntax_p0/mv/ut/aggOnAggMV10.groovy | 3 +- .../nereids_syntax_p0/mv/ut/aggOnAggMV11.groovy | 3 +- .../nereids_syntax_p0/mv/ut/aggOnAggMV2.groovy | 3 +- .../nereids_syntax_p0/mv/ut/aggOnAggMV3.groovy | 3 +- .../nereids_syntax_p0/mv/ut/aggOnAggMV6.groovy | 3 +- .../nereids_syntax_p0/mv/ut/aggOnAggMV7.groovy | 2 +- .../nereids_syntax_p0/mv/ut/bitmapUnionIn.groovy | 2 +- .../nereids_syntax_p0/mv/ut/incMVReInSub.groovy | 3 +- .../nereids_syntax_p0/mv/ut/incRewriteCD.groovy | 3 +- .../mv/ut/joinOnLeftPToJoin.groovy | 3 +- .../nereids_syntax_p0/mv/ut/orderByOnPView.groovy | 3 +- .../nereids_syntax_p0/mv/ut/projectMV1.groovy | 3 +- .../nereids_syntax_p0/mv/ut/projectMV2.groovy | 3 +- .../nereids_syntax_p0/mv/ut/projectMV3.groovy | 1 + .../nereids_syntax_p0/mv/ut/projectMV4.groovy | 3 +- .../suites/nereids_syntax_p0/mv/ut/subQuery.groovy | 3 +- .../suites/nereids_syntax_p0/mv/ut/unionDis.groovy | 3 +- .../suites/nereids_syntax_p0/rollup/hll/hll.groovy | 3 +- .../hll_with_light_sc/hll_with_light_sc.groovy | 3 +- .../suites/rollup_p0/test_materialized_view.groovy | 66 +- .../test_dup_mv_schema_change.groovy | 4 +- .../test_uniq_mv_schema_change.groovy | 31 +- .../workload_manager_p0/test_resource_tag.groovy | 99 --- 237 files changed, 2395 insertions(+), 1303 deletions(-) create mode 100644 fe/fe-common/src/main/java/org/apache/doris/common/security/authentication/PreExecutionAuthenticatorCache.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushDownFilterIntoSchemaScan.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/datasource/jdbc/client/JdbcClickHouseClientTest.java rename fe/fe-core/src/test/java/org/apache/doris/datasource/jdbc/{ => client}/JdbcClientExceptionTest.java (97%) copy fe/fe-core/src/test/java/org/apache/doris/{analysis/IsNullPredicateTest.java => datasource/paimon/PaimonExternalCatalogTest.java} (59%) create mode 100644 regression-test/data/correctness_p0/test_column_prune_in_hash_join.out create mode 100644 regression-test/data/information_schema_p0/test_information_schema.out copy regression-test/data/{workload_manager_p0/test_check_wg.out => mv_p0/agg_use_key_direct/agg_use_key_direct.out} (55%) copy regression-test/data/mv_p0/{no_await/no_await.out => await/await.out} (99%) delete mode 100644 regression-test/data/workload_manager_p0/skip_rg_test_table.csv create mode 100644 regression-test/suites/correctness_p0/test_column_prune_in_hash_join.groovy delete mode 100644 regression-test/suites/external_table_p2/es/test_external_catalog_es.groovy delete mode 100644 regression-test/suites/external_table_p2/es/test_external_es.groovy create mode 100644 regression-test/suites/information_schema_p0/test_information_schema.groovy create mode 100644 regression-test/suites/mv_p0/agg_use_key_direct/agg_use_key_direct.groovy copy regression-test/suites/mv_p0/{no_await/no_await.groovy => await/await.groovy} (65%) delete mode 100644 regression-test/suites/workload_manager_p0/test_resource_tag.groovy --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org