This is an automated email from the ASF dual-hosted git repository.
xiangfu0 pushed a change to branch xiangfu0/data-3221-10-physical-column-names
in repository https://gitbox.apache.org/repos/asf/pinot.git
discard 53a92fcb3ec DATA-3221 (12): make the new column-name accessor sorted
and null-safe
discard 37b08f28afb DATA-3221 (11): list physical columns without building the
segment schema
discard d8d960e9fef DATA-3221 (10): publish the segment column arrays as one
immutable pair
discard c68c4aefd8c DATA-3221 (10): hold segment column metadata in sorted
arrays, derive the map on demand
discard 5b0537d1065 DATA-3221 (8+9 follow-up): drop the shared shape, pack
bitsPerElement instead
discard ba484c280e0 DATA-3221 (8+9): shared column shape and primitive min/max
in ColumnMetadataImpl
discard ff260bad45f Slim ColumnMetadataImpl to 72 bytes and derive the
per-segment Schema lazily
discard fd9f74cc789 Document that segment-derived FieldSpecs and per-segment
Schemas are shared
discard 6cd8db36dfd Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
discard d728a18abe4 Fold PhysicalColumnIndexContainer's IndexTypeMap into a
presence mask plus dense readers
discard a78351ce939 Delegate immutable DataSourceMetadata to ColumnMetadata
instead of snapshotting it
add 3ae199f90c6 [perf] Encode MSE segment lists at dispatch, optionally as
protobuf (#19568)
add 9f3fa3a834a Run Apache RAT once from the root instead of per module
(#19650)
add 257a0e7df2c Reuse processed table state across segment loads (#19571)
add eaba6c92070 Delegate immutable DataSourceMetadata to ColumnMetadata
instead of snapshotting it
add ce3f3fe9c15 Trim duplicate data source metadata adapter tests
add f71886b640d Fold PhysicalColumnIndexContainer's IndexTypeMap into a
presence mask plus dense readers
add 66306c77cc0 Use ordered index ids and keep focused container coverage
add 28611868565 Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
add 460d1c123c6 Reuse resolved index configs for lazy column
materialization
add c633d4d078d Retain compact index configs for segment-specific loads
add 40d9a0b1317 Slim ColumnMetadataImpl to 72 bytes and derive the
per-segment Schema lazily
add b5e770c1c50 Use per-segment schema state in lazy metadata tests
add ece0ddd0a2d DATA-3221 (8+9): shared column shape and primitive min/max
in ColumnMetadataImpl
add 4c8cdd9e0d3 DATA-3221 (8+9 follow-up): drop the shared shape, pack
bitsPerElement instead
add 14df55d0a52 Keep focused packed metadata coverage
add 64f77154446 DATA-3221 (10): hold segment column metadata in sorted
arrays, derive the map on demand
add 1025203f334 DATA-3221 (10): publish the segment column arrays as one
immutable pair
add 088d63a41ea Use existing column views instead of extra metadata APIs
add 82d4b5b4ead Adapt lazy loader and config test to column views
add 3c83a4c8093 DATA-3221 (11): list physical columns without building the
segment schema
add e0f7a92a761 DATA-3221 (12): make the new column-name accessor sorted
and null-safe
add 6345b17ab17 Iterate column metadata directly during forward-index
planning
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (53a92fcb3ec)
\
N -- N -- N
refs/heads/xiangfu0/data-3221-10-physical-column-names (6345b17ab17)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../broker/broker/helix/BaseBrokerStarter.java | 3 +
.../MultiStageBrokerRequestHandler.java | 4 +-
.../metadata/segment/SegmentZKMetadataUtils.java | 6 +-
pinot-common/src/main/proto/worker.proto | 16 +
.../controller/utils/SegmentMetadataMockUtils.java | 8 -
.../core/data/manager/BaseTableDataManager.java | 22 +-
.../manager/offline/OfflineTableDataManager.java | 3 +-
.../manager/realtime/RealtimeTableDataManager.java | 12 +-
.../data/manager/BaseTableDataManagerTest.java | 152 +++++++
.../tests/MultiStageEngineIntegrationTest.java | 52 +++
.../BaseLogicalTableIntegrationTest.java | 45 ++
.../routing/BenchmarkSegmentListEncoding.java | 133 ++++++
.../planner/physical/DispatchablePlanContext.java | 14 +-
.../planner/physical/PinotDispatchPlanner.java | 2 +
.../pinot/query/routing/QueryPlanSerDeUtils.java | 121 +++++-
.../apache/pinot/query/routing/WorkerMetadata.java | 106 +++--
.../query/routing/QueryPlanSerDeUtilsTest.java | 239 +++++++++++
.../query/service/dispatch/QueryDispatcher.java | 72 +++-
.../pinot/query/service/server/QueryServer.java | 10 +-
.../service/dispatch/QueryDispatcherTest.java | 49 +++
.../query/service/server/QueryServerAuthzTest.java | 2 +-
.../query/service/server/QueryServerTest.java | 21 +-
.../indexsegment/immutable/ColumnMaterializer.java | 9 +-
.../immutable/ImmutableSegmentImpl.java | 15 +-
.../immutable/ImmutableSegmentLoader.java | 10 +-
.../index/column/PhysicalColumnIndexContainer.java | 4 +-
.../segment/index/loader/ForwardIndexHandler.java | 9 +-
.../segment/index/loader/IndexLoadingConfig.java | 461 +++++++++++++--------
.../PartitionIdVirtualColumnProvider.java | 7 +-
.../immutable/ColumnMaterializerTest.java | 41 +-
.../immutable/ImmutableSegmentImplTest.java | 14 +-
.../immutable/MockSegmentMetadata.java | 8 -
.../segment/index/SegmentMetadataImplTest.java | 71 +---
.../column/PhysicalColumnIndexContainerTest.java | 410 +-----------------
.../index/datasource/ImmutableDataSourceTest.java | 59 +--
.../index/loader/ForwardIndexHandlerTest.java | 2 +
.../index/loader/IndexLoadingConfigTest.java | 81 +++-
.../local/segment/index/loader/LoaderTest.java | 8 +-
.../index/map/ImmutableMapDataSourceTest.java | 6 +-
.../apache/pinot/segment/spi/ColumnMetadata.java | 10 -
.../apache/pinot/segment/spi/SegmentMetadata.java | 53 +--
.../pinot/segment/spi/index/FieldIndexConfigs.java | 7 +-
.../spi/index/metadata/SegmentMetadataImpl.java | 65 +--
.../segment/spi/SegmentMetadataDefaultsTest.java | 87 ----
.../spi/index/metadata/ColumnMetadataImplTest.java | 42 +-
.../pinot/server/api/resources/TablesResource.java | 2 +-
.../SegmentCompressionStatsReaderTest.java | 3 +-
.../apache/pinot/spi/utils/CommonConstants.java | 15 +
pom.xml | 20 +-
49 files changed, 1488 insertions(+), 1123 deletions(-)
create mode 100644
pinot-perf/src/main/java/org/apache/pinot/query/routing/BenchmarkSegmentListEncoding.java
create mode 100644
pinot-query-planner/src/test/java/org/apache/pinot/query/routing/QueryPlanSerDeUtilsTest.java
delete mode 100644
pinot-segment-spi/src/test/java/org/apache/pinot/segment/spi/SegmentMetadataDefaultsTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
