This is an automated email from the ASF dual-hosted git repository.
xiangfu0 pushed a change to branch
xiangfu0/data-3221-7-slim-metadata-lazy-schema
in repository https://gitbox.apache.org/repos/asf/pinot.git
discard ff260bad45f Slim ColumnMetadataImpl to 72 bytes and derive the
per-segment Schema lazily
discard fd9f74cc789 Document that segment-derived FieldSpecs and per-segment
Schemas are shared
discard 6cd8db36dfd Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
discard d728a18abe4 Fold PhysicalColumnIndexContainer's IndexTypeMap into a
presence mask plus dense readers
discard a78351ce939 Delegate immutable DataSourceMetadata to ColumnMetadata
instead of snapshotting it
add 3ae199f90c6 [perf] Encode MSE segment lists at dispatch, optionally as
protobuf (#19568)
add 9f3fa3a834a Run Apache RAT once from the root instead of per module
(#19650)
add 257a0e7df2c Reuse processed table state across segment loads (#19571)
add eaba6c92070 Delegate immutable DataSourceMetadata to ColumnMetadata
instead of snapshotting it
add ce3f3fe9c15 Trim duplicate data source metadata adapter tests
add f71886b640d Fold PhysicalColumnIndexContainer's IndexTypeMap into a
presence mask plus dense readers
add 66306c77cc0 Use ordered index ids and keep focused container coverage
add 28611868565 Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
add 460d1c123c6 Reuse resolved index configs for lazy column
materialization
add c633d4d078d Retain compact index configs for segment-specific loads
add 40d9a0b1317 Slim ColumnMetadataImpl to 72 bytes and derive the
per-segment Schema lazily
add b5e770c1c50 Use per-segment schema state in lazy metadata tests
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ff260bad45f)
\
N -- N -- N
refs/heads/xiangfu0/data-3221-7-slim-metadata-lazy-schema (b5e770c1c50)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../broker/broker/helix/BaseBrokerStarter.java | 3 +
.../MultiStageBrokerRequestHandler.java | 4 +-
pinot-common/src/main/proto/worker.proto | 16 +
.../core/data/manager/BaseTableDataManager.java | 22 +-
.../manager/offline/OfflineTableDataManager.java | 3 +-
.../manager/realtime/RealtimeTableDataManager.java | 12 +-
.../data/manager/BaseTableDataManagerTest.java | 152 +++++++
.../tests/MultiStageEngineIntegrationTest.java | 52 +++
.../BaseLogicalTableIntegrationTest.java | 45 ++
.../routing/BenchmarkSegmentListEncoding.java | 133 ++++++
.../planner/physical/DispatchablePlanContext.java | 14 +-
.../planner/physical/PinotDispatchPlanner.java | 2 +
.../pinot/query/routing/QueryPlanSerDeUtils.java | 121 +++++-
.../apache/pinot/query/routing/WorkerMetadata.java | 106 +++--
.../query/routing/QueryPlanSerDeUtilsTest.java | 239 +++++++++++
.../query/service/dispatch/QueryDispatcher.java | 72 +++-
.../pinot/query/service/server/QueryServer.java | 10 +-
.../service/dispatch/QueryDispatcherTest.java | 49 +++
.../query/service/server/QueryServerAuthzTest.java | 2 +-
.../query/service/server/QueryServerTest.java | 21 +-
.../indexsegment/immutable/ColumnMaterializer.java | 9 +-
.../immutable/ImmutableSegmentLoader.java | 8 +-
.../index/column/PhysicalColumnIndexContainer.java | 4 +-
.../segment/index/loader/IndexLoadingConfig.java | 461 +++++++++++++--------
.../immutable/ColumnMaterializerTest.java | 41 +-
.../immutable/ImmutableSegmentImplTest.java | 14 +-
.../segment/index/SegmentMetadataImplTest.java | 8 -
.../column/PhysicalColumnIndexContainerTest.java | 410 +-----------------
.../index/datasource/ImmutableDataSourceTest.java | 59 +--
.../index/loader/IndexLoadingConfigTest.java | 80 +++-
.../local/segment/index/loader/LoaderTest.java | 8 +-
.../index/map/ImmutableMapDataSourceTest.java | 6 +-
.../apache/pinot/segment/spi/ColumnMetadata.java | 10 -
.../apache/pinot/segment/spi/SegmentMetadata.java | 10 -
.../pinot/segment/spi/index/FieldIndexConfigs.java | 7 +-
.../spi/index/metadata/SegmentMetadataImpl.java | 13 -
.../apache/pinot/spi/utils/CommonConstants.java | 15 +
pom.xml | 20 +-
38 files changed, 1434 insertions(+), 827 deletions(-)
create mode 100644
pinot-perf/src/main/java/org/apache/pinot/query/routing/BenchmarkSegmentListEncoding.java
create mode 100644
pinot-query-planner/src/test/java/org/apache/pinot/query/routing/QueryPlanSerDeUtilsTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
