This is an automated email from the ASF dual-hosted git repository.
xiangfu0 pushed a change to branch
xiangfu0/data-3221-7-slim-metadata-lazy-schema
in repository https://gitbox.apache.org/repos/asf/pinot.git
discard 49a54b2cbc2 Invalidate the derived schema when the loader registers
virtual columns
discard ed2bf68149e Use per-segment schema state in lazy metadata tests
discard 3c0bd9b5b00 Slim ColumnMetadataImpl to 72 bytes and derive the
per-segment Schema lazily
discard 4ed98b81c17 Serialize v1/v2 index buffer access for lazy
materialization and cover destroy vs in-flight
discard 0133ad90ce6 Retain compact index configs for segment-specific loads
discard d00c2738a5e Reuse resolved index configs for lazy column
materialization
discard a802950ff7e Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
omit 1a1847a79e4 Use ordered index ids and keep focused container coverage
omit 33ba69ab7f6 Fold PhysicalColumnIndexContainer's IndexTypeMap into a
presence mask plus dense readers
add da63a4de680 Restore integration test selection broken by the Surefire
provider change (#19651)
add 39ef02cc5f2 Bump org.apache.httpcomponents.core5:httpcore5 from 5.4.3
to 5.4.4 (#19659)
add 36a3253a2d6 Run the integration test subpackages that no CI lane ever
selected (#19652)
add bd1e8824a88 Split list configs on commas when set via cluster config,
and trim planner rule names (#19644)
add d4365386d8b Fix realtime table deletion races with segment creation
(#19468)
add 33db247828c Bump com.fasterxml.jackson:jackson-bom from 2.22.2 to
2.22.3 (#19669)
add 024f506297b Bump org.apache.fory:fory-json from 1.7.4 to 1.7.5 (#19671)
add 8a6e620139c Bump software.amazon.awssdk:bom from 2.55.1 to 2.55.2
(#19670)
add 5985b69eacf Split pinot.broker.mse.planner.disabled.rules on commas
when set via cluster config (#19642)
add ff069f8ca85 Make the mutable dictionary pluggable through the
dictionary index type (#19656)
add f626b0c320a Bump junit-platform.version from 1.14.4 to 6.1.3 (#19658)
add 79b4354abcb Add -searchRecursively flag to CreateSegment CLI command
(#19609)
add 03d7ffd8f4d Send the table config and schema refresh message when a
schema is overridden (#19677)
add b4f9fe180bf Make BaseTableDataManagerTest register its ServerMetrics
mock regardless of prior registrations (#19678)
add 00e01874761 Keep tied ExprMin/ExprMax rows from every server when
merging serialized results (#19645)
add 8bfe71bbf47 Move polymorphic aggregate construction into per-family
providers (#19646)
add a88702102c2 Fold PhysicalColumnIndexContainer's IndexTypeMap into a
presence mask and a dense reader array (#19475)
add 3b630ad8a92 Materialize immutable-segment columns lazily behind an
opt-in instance config (default off)
add 43aee25f62a Reuse resolved index configs for lazy column
materialization
add d8f0e3cef25 Retain compact index configs for segment-specific loads
add 0f17b6301fb Serialize v1/v2 index buffer access for lazy
materialization and cover destroy vs in-flight
add 1199bcaa7e7 Slim ColumnMetadataImpl to 72 bytes and derive the
per-segment Schema lazily
add 069d076a859 Use per-segment schema state in lazy metadata tests
add 64382186301 Invalidate the derived schema when the loader registers
virtual columns
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (49a54b2cbc2)
\
N -- N -- N
refs/heads/xiangfu0/data-3221-7-slim-metadata-lazy-schema (64382186301)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../pr-tests/.pinot_tests_custom_integration.sh | 33 --
.../scripts/pr-tests/.pinot_tests_integration.sh | 67 ---
.../pr-tests/.pinot_tests_kinesis_integration.sh | 33 --
.../broker/broker/helix/BaseBrokerStarter.java | 8 +-
.../MultiStageBrokerRequestHandler.java | 17 +-
.../broker/broker/helix/BaseBrokerStarterTest.java | 53 ++
.../MultiStageBrokerRequestHandlerTest.java | 22 +
.../common/utils/config/QueryOptionsUtils.java | 26 +-
.../common/utils/config/QueryOptionsUtilsTest.java | 14 +
.../apache/pinot/controller/ControllerConf.java | 3 +-
.../helix/core/PinotHelixResourceManager.java | 50 +-
.../pinot/controller/ControllerConfTest.java | 20 +
.../PinotHelixResourceManagerStatelessTest.java | 32 ++
.../core/data/manager/BaseTableDataManager.java | 52 +-
.../core/data/manager/InstanceDataManager.java | 4 +-
.../manager/offline/DimensionTableDataManager.java | 4 +-
.../manager/offline/OfflineTableDataManager.java | 9 +-
.../realtime/RealtimeSegmentDataManager.java | 22 +-
.../manager/realtime/RealtimeTableDataManager.java | 126 ++++-
.../function/AggregationFunctionFactory.java | 151 +----
...ction.java => AggregationFunctionProvider.java} | 19 +-
.../AggregationFunctionProviderRegistry.java | 58 ++
.../function/AnyValueAggregationFunction.java | 14 +
.../ChildExprMinMaxAggregationFunction.java | 27 +
...rstLastWithTimeAggregationFunctionProvider.java | 114 ++++
.../function/ModeAggregationFunction.java | 14 +
.../ParentExprMinMaxAggregationFunction.java | 27 +
.../function/array/ArrayAggFunctionProvider.java | 107 ++++
.../utils/exprminmax/ExprMinMaxObject.java | 17 +-
...ggregation.function.AggregationFunctionProvider | 26 +
.../data/manager/BaseTableDataManagerTest.java | 139 +++++
.../realtime/RealtimeSegmentDataManagerTest.java | 56 +-
.../realtime/RealtimeTableDataManagerTest.java | 610 +++++++++++++++++++++
.../AggregationFunctionProviderRegistryTest.java | 58 ++
.../utils/exprminmax/ExprMinMaxObjectTest.java | 92 ++++
.../org/apache/pinot/queries/ExprMinMaxTest.java | 140 ++---
pinot-integration-tests/pom.xml | 313 ++++-------
.../BaseLogicalTableIntegrationTest.java | 228 +++++---
...alTableWithTwoRealtimeTableIntegrationTest.java | 40 --
...tomClusterSuite.java => LogicalTableSuite.java} | 18 +-
.../pinot/query/QueryPlannerRuleOptionsTest.java | 12 +
.../pinot/query/runtime/function/NotUdf.java | 2 +-
.../runtime/function/UdfServiceLoaderTest.java | 70 +++
.../indexsegment/mutable/MutableSegmentImpl.java | 3 +-
.../writer/impl/MutableOffHeapByteArrayStore.java | 135 +++--
.../dictionary/StringOffHeapMutableDictionary.java | 12 +-
.../index/column/PhysicalColumnIndexContainer.java | 42 +-
.../index/dictionary/DictionaryIndexType.java | 24 +-
.../impl/MutableOffHeapByteArrayStoreTest.java | 88 ++-
.../column/PhysicalColumnIndexContainerTest.java | 410 +++++++++++++-
.../pinot/segment/spi/index/IndexPlugin.java | 3 +
.../pinot/segment/spi/index/IndexService.java | 24 +
.../pinot/segment/spi/index/IndexServiceTest.java | 30 +
.../org/apache/pinot/server/conf/ServerConf.java | 4 +-
.../server/starter/helix/BaseServerStarter.java | 5 +
.../starter/helix/HelixInstanceDataManager.java | 82 ++-
.../apache/pinot/server/conf/ServerConfTest.java | 70 +++
.../HelixInstanceDataManagerLifecycleTest.java | 380 +++++++++++++
.../apache/pinot/spi/env/PinotConfiguration.java | 24 +
.../pinot/spi/env/PinotConfigurationTest.java | 30 +
.../tools/admin/command/CreateSegmentCommand.java | 23 +-
.../admin/command/CreateSegmentCommandTest.java | 107 ++++
pom.xml | 17 +-
63 files changed, 3500 insertions(+), 960 deletions(-)
delete mode 100755
.github/workflows/scripts/pr-tests/.pinot_tests_custom_integration.sh
delete mode 100755
.github/workflows/scripts/pr-tests/.pinot_tests_integration.sh
delete mode 100755
.github/workflows/scripts/pr-tests/.pinot_tests_kinesis_integration.sh
create mode 100644
pinot-broker/src/test/java/org/apache/pinot/broker/broker/helix/BaseBrokerStarterTest.java
copy
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/{BooleanOrAggregationFunction.java
=> AggregationFunctionProvider.java} (63%)
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionProviderRegistry.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/FirstLastWithTimeAggregationFunctionProvider.java
create mode 100644
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/array/ArrayAggFunctionProvider.java
create mode 100644
pinot-core/src/main/resources/META-INF/services/org.apache.pinot.core.query.aggregation.function.AggregationFunctionProvider
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionProviderRegistryTest.java
create mode 100644
pinot-core/src/test/java/org/apache/pinot/core/query/aggregation/utils/exprminmax/ExprMinMaxObjectTest.java
copy
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/suites/{CustomClusterSuite.java
=> LogicalTableSuite.java} (55%)
create mode 100644
pinot-query-runtime/src/test/java/org/apache/pinot/query/runtime/function/UdfServiceLoaderTest.java
create mode 100644
pinot-server/src/test/java/org/apache/pinot/server/conf/ServerConfTest.java
create mode 100644
pinot-server/src/test/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManagerLifecycleTest.java
create mode 100644
pinot-tools/src/test/java/org/apache/pinot/tools/admin/command/CreateSegmentCommandTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
