This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-catalog-spi
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 54943e403d0 [perf](catalog) fe-connector-iceberg hot-path caching +
fe-core per-statement metadata funnel, with session=user cache isolation
omit 442f23dc731 [fix](catalog) port #65676 iceberg deletion-vector
metadata validation to the connector scan path
omit f2ec03489b9 [fix](catalog) port #65548 external COUNT(*)/COUNT(col)
semantics to the connector scan path
omit 6e3191abd1f [fix](catalog) iceberg system-table scan: restore #65262
positional JNI read + order-preserving projection
omit d4231be7914 [fix](catalog) restore jfs hive-write hadoop config +
iceberg system-table scan projection
omit 034cec09f38 [feat](catalog) fe-connector-iceberg: port #64966 REST 401
re-auth to the connector
omit e1dba2a75fc [refactor](catalog) drop
ReauthenticatingRestSessionCatalog test orphaned by master rebase
omit 6ccae1ff9c7 [refactor](catalog) fe-core: remove 6 dead datasource
classes orphaned by the SPI migration (#65740)
omit 5990b8400bf [refactor](catalog) fe-core: remove dead
ExternalMetadataOps layer, deprecated LakeSoul catalog, and odbc/statistics
stubs (#65736)
omit fdda9928014 [refactor](catalog) fe-core: remove dead Trino-ported
DirectoryListercache cluster (#65734)
omit 0694b7fb78c [refactor](catalog) fe-connector: replace fat
hive-catalog-shade with a slim HMS metastore-client shade (#65721)
omit 661211c7f0a [refactor](catalog) Catalog spi 11 hive (#65473)
omit b5572c26cce [doc](catalog) HANDOFF: $position_deletes port done; e2e
is the only remaining flip gate
omit 4737f9485cc [feat](catalog) P6.6: support the iceberg
$position_deletes system table on the connector SPI
omit 3018112ca67 [doc](catalog) HANDOFF: point next step at the
$position_deletes port design (T0 evidence in)
omit 798dc7b7d67 [doc](catalog) P6.6 $position_deletes connector port:
research notes + design (T0 evidence in, ready to implement)
omit 01c09fc789b [doc](catalog) HANDOFF: rebase to upstream 30a44d61a8a +
register $position_deletes connector gap as flip blocker
omit a25f2f0b9a2 [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
omit 674eb26f91c [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
omit 1e7b3063ea5 [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
omit 58a829b7c92 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
omit 061cabdcad9 [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
omit 1d4c6505777 [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
omit 7637ef4046e [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
omit cf703220183 [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
omit 897723de723 [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
omit 7f8c4dc9e02 [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
omit b3821d28a30 [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
omit c28a4fea547 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
omit 750bf1018c5 [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
omit 57128e6210e [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
omit a0aabc37638 [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
omit fd59353f09c [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
omit 0dfda9eb416 update P5 handoff and fix compile issue
omit 4c4904ec86f [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
omit e92ed372e91 [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
omit c8205e5f3f6 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
omit edcc3dc3332 [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
omit 52d70c63344 [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
omit 84b62c7e83b [doc](connector) add project tracking system for catalog
SPI migration
add d2a3b48a9c8 [fix](streaming-job) Isolate schemas for CDC snapshot
splits (#65645)
add 8460676f3fc [Fix](nereids) Freeze sortedPartitionRanges in
SelectedPartitions to prevent TOCTOU NPE during partition pruning (#65659)
add 181e0d640c0 [test](regression) Add Iceberg v3 row lineage coverage
(#63719)
add a7951aac9fe [doc](connector) add project tracking system for catalog
SPI migration
add 0dbbe4b0b9c [feat](connector) P0 SPI baseline + DDL/Partition + import
gate (T03-T27) (#63582)
add 11b12397955 [P1-T03-T05] route plugin-driven scans first in nereids
translator (#63641)
add d78b6e0a2c8 [feat](connector) P2 migrate trino-connector to catalog
SPI (T01-T13) (#64096)
add 1f7a53bb67a [feat](connector) P3 hudi connector hardening + test
baseline + dispatch design (hybrid, T02-T08) (#64143)
add d3c517b15d1 [refactor](connector) P4 maxcompute: remove legacy
subsystem from fe-core + make fe-core odps-free (T07-T09) (#64300)
add 26917f3a62a update P5 handoff and fix compile issue
add 494183213c8 [refactor](catalog) P5 paimon: migrate to catalog SPI +
cutover (#64446) (#64446)
add 186f0255631 [refactor](catalog) P5 paimon: remove legacy subsystem
from fe-core + make fe-core paimon-SDK-free (T29) (#64653)
add 290911fbc3c [refactor](catalog) P3b: consolidate kerberos
authentication into fe-kerberos (#64655)
add 1eceb20b70e [refactor](catalog) P6 iceberg: migrate to catalog SPI +
cutover + remove legacy fe-core subsystem (#64688)
add 22e3aa10935 [fix](catalog) jdbc: keep driver classloaders alive per
URL to stop Metaspace leak
add 9ea01f46038 [fix](catalog) P4 maxcompute: drop dangling legacy
MaxComputeExternalTableTest after rebase
add 6f970e40c3b [fix](catalog) port upstream #65332/#65094 datasource
fixes into SPI connectors
add 62da3ee7157 [fix](catalog) tier3 #65094: preserve external-table
top-level column-name case on the read path
add 734610cc2cb [feat](catalog) P6.6-C6 #63068 SPI: neutral
delegated-credential DTO + user-session capability
add 9b77fad0ff8 [feat](catalog) P6.6-C6 #63068 FE: capability-gated
credential injection + per-user cache bypass
add 5d8bcce16b2 [feat](catalog) P6.6-C6 #63068 iceberg: per-user REST
session routing (metadata + scan/write/procedure)
add 83578743c19 [doc](catalog) P6.6-C6 #63068 OIDC session migration:
design + research notes + HANDOFF (Tasks 0-7 DONE)
add f4fb8d3d708 [feat](catalog) P6.6-C6 #63068 FE: re-migrate
ExternalDatabaseSessionContextTest (cross-user cache-leakage data-flow)
add a788e36a30d [doc](catalog) P6.6-C6 #63068 HANDOFF: record
cache-leakage data-flow test + Task 0 retained-base verify
add 68ac95c46ba [fix](catalog) reconcile upstream #65365 with connector
SPI after rebase to 4f48ccfaa7d
add 349effe9a9c [fix](build) fix parallel -T shade race in
fe-connector-paimon-hive-shade
add fb80955876f [doc](catalog) HANDOFF: rebase to upstream 30a44d61a8a +
register $position_deletes connector gap as flip blocker
add 0adf8287dfa [doc](catalog) P6.6 $position_deletes connector port:
research notes + design (T0 evidence in, ready to implement)
add 23cf09a4fcb [doc](catalog) HANDOFF: point next step at the
$position_deletes port design (T0 evidence in)
add ab1762d04fd [feat](catalog) P6.6: support the iceberg
$position_deletes system table on the connector SPI
add ad3d6f73b3a [doc](catalog) HANDOFF: $position_deletes port done; e2e
is the only remaining flip gate
add 3593684715f [refactor](catalog) Catalog spi 11 hive (#65473)
add 902eec59ef0 [refactor](catalog) fe-connector: replace fat
hive-catalog-shade with a slim HMS metastore-client shade (#65721)
add f4529a9133c [refactor](catalog) fe-core: remove dead Trino-ported
DirectoryListercache cluster (#65734)
add 677a2326745 [refactor](catalog) fe-core: remove dead
ExternalMetadataOps layer, deprecated LakeSoul catalog, and odbc/statistics
stubs (#65736)
add 2535c250f40 [refactor](catalog) fe-core: remove 6 dead datasource
classes orphaned by the SPI migration (#65740)
add f61d42c2247 [refactor](catalog) drop
ReauthenticatingRestSessionCatalog test orphaned by master rebase
add 031ddd5a804 [feat](catalog) fe-connector-iceberg: port #64966 REST 401
re-auth to the connector
add 707f438f81e [fix](catalog) restore jfs hive-write hadoop config +
iceberg system-table scan projection
add 11f4deaa508 [fix](catalog) iceberg system-table scan: restore #65262
positional JNI read + order-preserving projection
add e697837760d [fix](catalog) port #65548 external COUNT(*)/COUNT(col)
semantics to the connector scan path
add 1ea735ff0a5 [fix](catalog) port #65676 iceberg deletion-vector
metadata validation to the connector scan path
add 777a61671ab [perf](catalog) fe-connector-iceberg hot-path caching +
fe-core per-statement metadata funnel, with session=user cache isolation
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (54943e403d0)
\
N -- N -- N refs/heads/branch-catalog-spi (777a61671ab)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../docker-compose/iceberg/entrypoint.sh.tpl | 6 +-
.../rules/rewrite/PruneFileScanPartition.java | 14 +-
.../trees/plans/logical/LogicalFileScan.java | 29 +-
.../BinarySearchPartitionInconsistencyTest.java | 255 +++++++++++++++++
.../source/reader/JdbcIncrementalSourceReader.java | 2 +-
.../source/reader/mysql/MySqlSourceReader.java | 2 +-
.../source/reader/mysql/MySqlSourceReaderTest.java | 73 +++++
.../reader/postgres/PostgresSourceReaderTest.java | 74 +++++
...ming_mysql_job_snapshot_with_concurrent_dml.out | 23 ++
...est_iceberg_v2_to_v3_doris_spark_compare.groovy | 239 ++++++++++++++--
...ceberg_v3_doris_create_update_spark_read.groovy | 170 +++++++++++
...est_iceberg_v3_row_lineage_complex_query.groovy | 219 ++++++++++++++
...st_iceberg_v3_row_lineage_continuous_dml.groovy | 274 ++++++++++++++++++
...t_iceberg_v3_row_lineage_dml_mode_matrix.groovy | 279 ++++++++++++++++++
...3_row_lineage_equality_delete_dv_interop.groovy | 317 +++++++++++++++++++++
...test_iceberg_v3_row_lineage_query_insert.groovy | 222 ++++++++++++++-
..._iceberg_v3_row_lineage_schema_evolution.groovy | 234 +++++++++++++++
...rg_v3_row_lineage_spark_doris_dv_interop.groovy | 294 +++++++++++++++++++
.../test_iceberg_v3_row_lineage_time_travel.groovy | 217 ++++++++++++++
...berg_v3_row_lineage_uniqueness_stability.groovy | 188 ++++++++++++
...eberg_v3_row_lineage_update_delete_merge.groovy | 149 ++++++++--
...t_iceberg_v3_row_lineage_large_stability.groovy | 169 +++++++++++
...eberg_v3_row_lineage_random_cross_engine.groovy | 270 ++++++++++++++++++
...g_mysql_job_snapshot_with_concurrent_dml.groovy | 95 ++++--
...snapshot_with_concurrent_dml_multi_table.groovy | 71 +++--
25 files changed, 3791 insertions(+), 94 deletions(-)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/BinarySearchPartitionInconsistencyTest.java
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_doris_create_update_spark_read.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_row_lineage_complex_query.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_row_lineage_continuous_dml.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_row_lineage_dml_mode_matrix.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_row_lineage_equality_delete_dv_interop.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_row_lineage_schema_evolution.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_row_lineage_spark_doris_dv_interop.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_row_lineage_time_travel.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_v3_row_lineage_uniqueness_stability.groovy
create mode 100644
regression-test/suites/external_table_p2/iceberg/test_iceberg_v3_row_lineage_large_stability.groovy
create mode 100644
regression-test/suites/external_table_p2/iceberg/test_iceberg_v3_row_lineage_random_cross_engine.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
