This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a change to branch auto-pick-42904-branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 99f58c7a0c0 [test](doc) add job-scheduler example in doris's doc to
regression test (#42904)
add aa496914ff7 branch-3.0: [opt](cloud) Make get tablet stats and update
delete bitmap update lock be able to be in different fdb txns #45206 (#45559)
add c04a8da75bc branch-3.0: [fix](vault) Fix bugs about altering storage
vault name #45685 (#45963)
add 9f2097c882e branch-3.0: [fix](cloud) fix abort transaction in
runningTxns list when show routine load #45629 (#45721)
add 4ea5b16105e branch-3.0: [fix](meta-mgr) Real-time parsing meta service
endpoint to avoid rpc failed after config muted #45877 (#45899)
add af2d6287611 branch-3.0: [enhancement](cloud) add profile counter for
file cache #45678 (#45892)
add d729009c95d branch-3.0: [fix](csv reader) fix core dump when parsing
csv with enclose #45485 (#45888)
add 3a1743a1109 branch-3.0: [fix](olap) Set the original tablet state to
TABLET_SHUTDOWN #45812 (#45891)
add 44cbeaca577 branch-3.0: [enhancement](meta) Sync tablet meta even if
local state is not running #45821 (#45962)
add 47050877928 brnahc-3.0:[improvement](jdbc catalog) Optimize
JdbcCatalog case mapping stability (#45221)
add 0cfdf4bd290 branch-3.0: [feature](table-function)support posexplode
table function #43221 (#45782)
add 3c5797bd99c branch-3.0: [fix](mem leak) fe non_heap mem leak while use
jdbc catalog #45806 (#45979)
add b06cf95e32c branch-3.0: [fix](regression)Fix paimon row count case.
#45994 (#46005)
add ee733c0ec44 branch-3.0: [improvement](statistics)Async drop table
stats while doing truncate and schema change. #45923 (#45997)
add 86369bb2c3d branch-3.0: [enchement](mc)opt maxcompute read arrow skip
repeated check of isNull #45989 (#46022)
add 9a49926ee09 branch-3.0: [enhance](mtmv)When drop MTMV, no longer wait
for task cancel to complete #45995 (#46014)
add 24a23d7e976 branch-3.0: [fix](case)fix manager regression case. #45993
(#46032)
add fc0078e07ca branch-3.0: [fix](show)show tables should be case
insensitive when lowerCaseTableNames is 1 or 2. #46030 (#46057)
add cac25bec45a branch-3.0: [opt](bloomfilter index) optimize memory usage
for bloom filter index writer #45833 (#46048)
add 949ad8be043 branch-3.0: [opt](inverted index)Optimize code to get rid
of heap use after free #45745 (#46051)
add 4746f167892 branch-3.0: [test](index compaction) Add index compaction
full flow UT test #45746 (#46049)
add c4f8591652e branch-3.0: [fix](profile) Change the check of whether the
username is admin and root to check whether the user specifically corresponds
to admin permissions #41714 (#44864)
add 9e4a0a910c7 branch-3.0: [fix](code) remove unused files #46078 (#46080)
add 1cb1d94b87d branch-3.0: [test](regression)Fix unstable write inverted
index case #46046 (#46066)
add 64f30f3d8f9 branch-3.0: [fix](clone) Fix the skipped link file due to
the stale value #46009 (#46067)
add dfe7db7094c branch-3.0: [chore](binlog) Add table type to getMeta
method #45942 (#45971)
add 01dc5bcd276 [opt](hms table)Some optimizations for hms external table
for 3.0 (#44909) (#46086)
add bbef3ec6903 [enchement](utf8)import enable_text_validate_utf8 session
var (#45537) (#46071)
add 64578dafcb4 branch-3.0: [test](nereids) add arthmetic comparison ut
#45690 (#45894)
add 06b7485e5ee branch-3.0: [fix](variant) disable column name with dot
character for variant type #45927 (#45990)
add c36eead3063 branch-3.0: [fix](regression test) fix stream load 2pc
case fail #45919 (#45975)
add bf74831f46c branch-3.0: [regression-test](fix) remove interrupt() in
test_routine_load_eof.groovy #46044 (#46097)
add 1e006eecbe0 [test](doc) add job-scheduler example in doris's doc to
regression test (#42904)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (99f58c7a0c0)
\
N -- N -- N refs/heads/auto-pick-42904-branch-3.0 (1e006eecbe0)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
aazcp.tar.gz | Bin 4218 -> 0 bytes
be/src/cloud/cloud_meta_mgr.cpp | 17 +-
be/src/cloud/cloud_tablet.cpp | 3 -
be/src/cloud/cloud_tablet_mgr.cpp | 3 -
be/src/io/cache/block_file_cache.cpp | 20 +-
be/src/io/cache/block_file_cache_profile.h | 17 +
be/src/io/cache/cached_remote_file_reader.cpp | 11 +
be/src/io/cache/cached_remote_file_reader.h | 9 -
be/src/io/cache/file_cache_common.cpp | 2 +
be/src/io/cache/file_cache_common.h | 16 +
be/src/io/io_common.h | 5 +
be/src/olap/rowset/segment_v2/bloom_filter.h | 10 +
.../segment_v2/bloom_filter_index_writer.cpp | 27 +-
.../segment_v2/inverted_index_compound_reader.cpp | 17 +
.../segment_v2/inverted_index_compound_reader.h | 1 +
.../segment_v2/inverted_index_fs_directory.cpp | 4 +-
.../rowset/segment_v2/inverted_index_writer.cpp | 12 +-
be/src/olap/tablet_manager.cpp | 32 +-
be/src/olap/task/engine_clone_task.cpp | 4 +-
be/src/util/utf8_check.cpp | 7 +
be/src/util/utf8_check.h | 4 +
be/src/vec/exec/format/csv/csv_reader.cpp | 6 +-
.../file_reader/new_plain_text_line_reader.cpp | 3 +-
.../table_function/table_function_factory.cpp | 2 +
be/src/vec/exprs/table_function/vposexplode.cpp | 155 ++++
be/src/vec/exprs/table_function/vposexplode.h | 50 ++
be/src/vec/functions/function_fake.cpp | 29 +
be/test/io/cache/block_file_cache_test.cpp | 146 ++++
.../bloom_filter_index_reader_writer_test.cpp | 7 +-
.../index_compaction_performance_test.cpp | 265 ++++++
.../compaction/index_compaction_test.cpp | 912 ++++++++++++++++++++-
.../compaction/util/index_compaction_utils.cpp | 275 +++++--
.../inverted_index/data/sorted_wikipedia-50-1.json | 50 ++
.../inverted_index/data/sorted_wikipedia-50-2.json | 50 ++
be/test/olap/tablet_mgr_test.cpp | 45 +
build_plugin.sh => build-plugin.sh | 0
cloud/src/meta-service/meta_service.cpp | 98 ++-
cloud/src/meta-service/meta_service_resource.cpp | 55 +-
cloud/test/meta_service_test.cpp | 315 ++++++-
.../scripts/create_preinstalled_scripts/run72.hql | 31 +
.../text/utf8_check/utf8_check_fail.csv | 5 +
.../doris/maxcompute/MaxComputeJniScanner.java | 8 +-
fe/check/checkstyle/suppressions.xml | 1 +
.../org/apache/doris/analysis/CreateTableStmt.java | 5 +
.../catalog/BuiltinTableGeneratingFunctions.java | 6 +-
.../main/java/org/apache/doris/catalog/Env.java | 6 +-
.../java/org/apache/doris/catalog/OlapTable.java | 20 +-
.../org/apache/doris/catalog/StorageVault.java | 3 +-
.../org/apache/doris/catalog/StorageVaultMgr.java | 52 +-
.../org/apache/doris/catalog/TableProperty.java | 4 -
.../cloud/datasource/CloudInternalCatalog.java | 36 +-
.../transaction/CloudGlobalTransactionMgr.java | 23 +-
.../org/apache/doris/datasource/CatalogIf.java | 10 +
.../org/apache/doris/datasource/CatalogMgr.java | 12 +-
.../apache/doris/datasource/ExternalCatalog.java | 196 ++++-
.../apache/doris/datasource/ExternalDatabase.java | 238 +++++-
.../doris/datasource/ExternalMetaCacheMgr.java | 33 +-
.../org/apache/doris/datasource/ExternalTable.java | 26 +-
.../apache/doris/datasource/InitCatalogLog.java | 7 +-
.../apache/doris/datasource/InitDatabaseLog.java | 7 +-
.../apache/doris/datasource/InternalCatalog.java | 1 -
.../doris/datasource/es/EsExternalDatabase.java | 12 +-
.../doris/datasource/es/EsExternalTable.java | 9 +-
.../doris/datasource/hive/HMSExternalCatalog.java | 2 +-
.../doris/datasource/hive/HMSExternalDatabase.java | 12 +-
.../doris/datasource/hive/HMSExternalTable.java | 56 +-
.../datasource/hive/HiveMetaStoreClientHelper.java | 20 +-
.../doris/datasource/hive/source/HiveScanNode.java | 4 +
.../datasource/hudi/HudiSchemaCacheValue.java | 30 +-
.../apache/doris/datasource/hudi/HudiUtils.java | 12 +-
.../hudi/source/HudiCachedFsViewProcessor.java | 134 +++
.../hudi/source/HudiCachedMetaClientProcessor.java | 152 ++++
.../hudi/source/HudiMetadataCacheMgr.java | 143 ++++
.../datasource/hudi/source/HudiPartitionMgr.java | 73 --
.../doris/datasource/hudi/source/HudiScanNode.java | 48 +-
.../iceberg/IcebergExternalDatabase.java | 11 +-
.../datasource/iceberg/IcebergExternalTable.java | 5 +-
.../infoschema/ExternalInfoSchemaDatabase.java | 8 +-
.../infoschema/ExternalInfoSchemaTable.java | 6 +-
.../infoschema/ExternalMysqlDatabase.java | 8 +-
.../datasource/infoschema/ExternalMysqlTable.java | 6 +-
.../doris/datasource/jdbc/JdbcExternalCatalog.java | 74 +-
.../datasource/jdbc/JdbcExternalDatabase.java | 11 +-
.../doris/datasource/jdbc/JdbcExternalTable.java | 91 +-
.../datasource/jdbc/JdbcIdentifierMapping.java | 45 -
.../datasource/jdbc/JdbcSchemaCacheValue.java | 27 +-
.../doris/datasource/jdbc/client/JdbcClient.java | 61 +-
.../datasource/jdbc/client/JdbcGbaseClient.java | 4 +-
.../datasource/jdbc/client/JdbcMySQLClient.java | 4 +-
.../datasource/jdbc/client/JdbcOracleClient.java | 4 +-
.../jdbc/client/JdbcPostgreSQLClient.java | 4 +-
.../lakesoul/LakeSoulExternalDatabase.java | 11 +-
.../datasource/lakesoul/LakeSoulExternalTable.java | 19 +-
.../datasource/mapping/IdentifierMapping.java | 311 +------
.../datasource/mapping/JdbcIdentifierMapping.java | 345 ++++++++
.../maxcompute/MaxComputeExternalDatabase.java | 12 +-
.../maxcompute/MaxComputeExternalTable.java | 7 +-
.../doris/datasource/metacache/MetaCache.java | 22 +-
.../datasource/paimon/PaimonExternalDatabase.java | 11 +-
.../datasource/paimon/PaimonExternalTable.java | 5 +-
.../datasource/test/TestExternalDatabase.java | 11 +-
.../doris/datasource/test/TestExternalTable.java | 4 +-
.../TrinoConnectorExternalDatabase.java | 12 +-
.../TrinoConnectorExternalTable.java | 5 +-
.../httpv2/rest/manager/QueryProfileAction.java | 2 +-
.../org/apache/doris/job/base/AbstractJob.java | 8 +-
.../main/java/org/apache/doris/job/base/Job.java | 2 +-
.../doris/job/executor/DispatchTaskHandler.java | 2 +-
.../doris/job/extensions/insert/InsertJob.java | 4 +-
.../doris/job/extensions/insert/InsertTask.java | 2 +-
.../apache/doris/job/extensions/mtmv/MTMVTask.java | 4 +-
.../org/apache/doris/job/task/AbstractTask.java | 8 +-
.../main/java/org/apache/doris/job/task/Task.java | 4 +-
.../functions/generator/PosExplode.java | 80 ++
.../functions/generator/PosExplodeOuter.java | 80 ++
.../visitor/TableGeneratingFunctionVisitor.java | 10 +
.../plans/commands/AlterStorageVaultCommand.java | 10 +
.../trees/plans/commands/info/CreateTableInfo.java | 5 +
.../trees/plans/logical/LogicalHudiScan.java | 2 +-
.../java/org/apache/doris/qe/SessionVariable.java | 10 +
.../java/org/apache/doris/qe/ShowExecutor.java | 13 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 8 +-
.../apache/doris/service/FrontendServiceImpl.java | 6 +-
.../apache/doris/statistics/AnalysisManager.java | 64 +-
.../doris/statistics/AnalysisTaskExecutor.java | 6 +-
.../doris/statistics/StatisticsAutoCollector.java | 2 +-
.../ExternalFileTableValuedFunction.java | 2 +
.../doris/tablefunction/MetadataGenerator.java | 7 +-
.../hadoop/hbase/io/FSDataInputStreamWrapper.java | 368 +++++++++
.../catalog/constraint/ConstraintPersistTest.java | 8 +-
.../apache/doris/datasource/CatalogMgrTest.java | 8 +-
.../datasource/hive/HiveDDLAndDMLPlanTest.java | 4 +-
.../doris/datasource/hive/HiveMetadataOpsTest.java | 2 +-
.../doris/datasource/hudi/HudiUtilsTest.java | 198 +++++
.../datasource/iceberg/CreateIcebergTableTest.java | 2 +-
...leNameComparedLowercaseMetaCacheFalseTest.java} | 21 +-
...bleNameComparedLowercaseMetaCacheTrueTest.java} | 21 +-
...ableNameStoredLowercaseMetaCacheFalseTest.java} | 21 +-
...TableNameStoredLowercaseMetaCacheTrueTest.java} | 21 +-
.../mapping/JdbcIdentifierMappingTest.java | 277 +++++++
.../apache/doris/external/hms/HmsCatalogTest.java | 2 +-
.../expression/ExpressionRewriteTestHelper.java | 3 +
.../SimplifyArithmeticComparisonRuleTest.java | 193 +++--
.../org/apache/doris/qe/HmsQueryCacheTest.java | 2 +-
.../java/org/apache/doris/qe/ShowExecutorTest.java | 13 +
.../doris/statistics/AnalysisManagerTest.java | 8 +-
.../statistics/StatisticsAutoCollectorTest.java | 10 +-
.../doris/statistics/util/StatisticsUtilTest.java | 19 +-
gensrc/thrift/FrontendService.thrift | 1 +
gensrc/thrift/PlanNodes.thrift | 2 +
.../external_table_p0/hive/test_utf8_check.out | 55 ++
.../lower_case/test_conflict_name.out | 5 +
.../test_lower_case_meta_show_and_select.out | 49 ++
..._meta_with_lower_table_conf_show_and_select.out | 241 ++++++
.../lower_case/test_lower_case_mtmv.out} | 0
.../test_meta_cache_select_without_refresh.out | 10 +
.../lower_case/test_meta_names_mapping.out | 13 +
.../external_table_p0/lower_case/upgrade/load.out | 7 +
.../upgrade/test_upgrade_lower_case_catalog.out | 7 +
...st_index_compound_directory_fault_injection.out | 22 -
.../test_csv_with_enclose_and_escapeS3_load.out | 2 +
.../test_csv_with_enclose_and_escape.out | 2 +
.../sql_functions/table_function/posexplode.out | 166 ++++
.../org/apache/doris/regression/suite/Suite.groovy | 4 +-
.../suites/ddl_p0/test_create_table.groovy | 30 +
.../external_table_p0/hive/test_utf8_check.groovy | 100 +++
.../lower_case/test_conflict_name.groovy | 94 +++
.../lower_case/test_lower_case_meta_include.groovy | 158 ++++
.../test_lower_case_meta_show_and_select.groovy | 254 ++++++
...ta_with_lower_table_conf_show_and_select.groovy | 702 ++++++++++++++++
.../lower_case/test_lower_case_mtmv.groovy | 64 ++
.../test_meta_cache_select_without_refresh.groovy | 92 +++
.../lower_case/test_meta_names_mapping.groovy | 289 +++++++
.../lower_case/test_timing_refresh_catalog.groovy | 161 ++++
.../lower_case/upgrade/load.groovy | 89 ++
.../upgrade/test_upgrade_lower_case_catalog.groovy | 47 ++
.../paimon/test_paimon_table_stats.groovy | 7 +-
...index_compound_directory_fault_injection.groovy | 190 +++--
...inverted_index_exception_fault_injection.groovy | 66 +-
.../test_http_permissions_check_auth.groovy | 66 ++
.../test_csv_with_enclose_and_escapeS3_load.groovy | 2 +-
.../routine_load/test_routine_load_eof.groovy | 1 -
.../test_csv_with_enclose_and_escape.groovy | 2 +-
.../stream_load/test_stream_load_properties.groovy | 6 +-
.../suites/manager/test_manager_interface_1.groovy | 12 +-
.../sql_functions/table_function/posexplode.groovy | 82 ++
.../suites/statistics/analyze_stats.groovy | 20 +
.../suites/statistics/test_analyze_mv.groovy | 17 +
.../statistics/test_drop_stats_and_truncate.groovy | 18 +
.../vault_p0/alter/test_alter_vault_name.groovy | 232 ++++++
190 files changed, 8863 insertions(+), 1331 deletions(-)
delete mode 100644 aazcp.tar.gz
create mode 100644 be/src/vec/exprs/table_function/vposexplode.cpp
create mode 100644 be/src/vec/exprs/table_function/vposexplode.h
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/compaction/index_compaction_performance_test.cpp
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/data/sorted_wikipedia-50-1.json
create mode 100644
be/test/olap/rowset/segment_v2/inverted_index/data/sorted_wikipedia-50-2.json
rename build_plugin.sh => build-plugin.sh (100%)
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/create_preinstalled_scripts/run72.hql
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/preinstalled_data/text/utf8_check/utf8_check_fail.csv
copy be/src/util/utf8_check.h =>
fe/fe-core/src/main/java/org/apache/doris/datasource/hudi/HudiSchemaCacheValue.java
(58%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/hudi/source/HudiCachedFsViewProcessor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/hudi/source/HudiCachedMetaClientProcessor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/hudi/source/HudiMetadataCacheMgr.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/hudi/source/HudiPartitionMgr.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcIdentifierMapping.java
copy be/src/util/utf8_check.h =>
fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcSchemaCacheValue.java
(57%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/mapping/JdbcIdentifierMapping.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/PosExplode.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/PosExplodeOuter.java
create mode 100644
fe/fe-core/src/main/java/org/apache/hadoop/hbase/io/FSDataInputStreamWrapper.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/hudi/HudiUtilsTest.java
copy
fe/fe-core/src/test/java/org/apache/doris/datasource/lowercase/{ExternalTableNameComparedLowercaseTest.java
=> ExternalTableNameComparedLowercaseMetaCacheFalseTest.java} (82%)
rename
fe/fe-core/src/test/java/org/apache/doris/datasource/lowercase/{ExternalTableNameComparedLowercaseTest.java
=> ExternalTableNameComparedLowercaseMetaCacheTrueTest.java} (82%)
copy
fe/fe-core/src/test/java/org/apache/doris/datasource/lowercase/{ExternalTableNameStoredLowercaseTest.java
=> ExternalTableNameStoredLowercaseMetaCacheFalseTest.java} (83%)
rename
fe/fe-core/src/test/java/org/apache/doris/datasource/lowercase/{ExternalTableNameStoredLowercaseTest.java
=> ExternalTableNameStoredLowercaseMetaCacheTrueTest.java} (83%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/mapping/JdbcIdentifierMappingTest.java
create mode 100644
regression-test/data/external_table_p0/hive/test_utf8_check.out
create mode 100644
regression-test/data/external_table_p0/lower_case/test_conflict_name.out
create mode 100644
regression-test/data/external_table_p0/lower_case/test_lower_case_meta_show_and_select.out
create mode 100644
regression-test/data/external_table_p0/lower_case/test_lower_case_meta_with_lower_table_conf_show_and_select.out
copy
regression-test/data/{correctness/test_constant_push_down_through_outer_join.out
=> external_table_p0/lower_case/test_lower_case_mtmv.out} (100%)
create mode 100644
regression-test/data/external_table_p0/lower_case/test_meta_cache_select_without_refresh.out
create mode 100644
regression-test/data/external_table_p0/lower_case/test_meta_names_mapping.out
create mode 100644
regression-test/data/external_table_p0/lower_case/upgrade/load.out
create mode 100644
regression-test/data/external_table_p0/lower_case/upgrade/test_upgrade_lower_case_catalog.out
delete mode 100644
regression-test/data/fault_injection_p0/test_index_compound_directory_fault_injection.out
create mode 100644
regression-test/data/nereids_p0/sql_functions/table_function/posexplode.out
create mode 100644
regression-test/suites/external_table_p0/hive/test_utf8_check.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/test_conflict_name.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/test_lower_case_meta_include.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/test_lower_case_meta_show_and_select.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/test_lower_case_meta_with_lower_table_conf_show_and_select.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/test_lower_case_mtmv.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/test_meta_cache_select_without_refresh.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/test_meta_names_mapping.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/test_timing_refresh_catalog.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/upgrade/load.groovy
create mode 100644
regression-test/suites/external_table_p0/lower_case/upgrade/test_upgrade_lower_case_catalog.groovy
create mode 100644
regression-test/suites/http_auth_check/test_http_permissions_check_auth.groovy
create mode 100644
regression-test/suites/nereids_p0/sql_functions/table_function/posexplode.groovy
create mode 100644
regression-test/suites/vault_p0/alter/test_alter_vault_name.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
