This is an automated email from the ASF dual-hosted git repository.
capistrant pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 2e61c7120a5 refactor: enforce 1 tierAlias mapping per distinct tier.
add tierAlias dim to some metrics (#19595)
2e61c7120a5 is described below
commit 2e61c7120a562656a44473034301d4d5a584ca4e
Author: Lucas Capistrant <[email protected]>
AuthorDate: Thu Jun 18 07:26:14 2026 -0500
refactor: enforce 1 tierAlias mapping per distinct tier. add tierAlias dim
to some metrics (#19595)
---
docs/configuration/index.md | 2 +-
docs/operations/metrics.md | 8 +-
.../coordinator/CoordinatorDynamicConfig.java | 45 +++++++
.../duty/PrepareBalancerAndLoadQueues.java | 14 ++-
.../coordinator/loading/SegmentLoadingConfig.java | 20 ++-
.../loading/StrategicSegmentAssigner.java | 18 ++-
.../druid/server/coordinator/stats/Dimension.java | 1 +
.../server/coordinator/rules/LoadRuleTest.java | 42 +++++++
.../simulate/HistoricalTierAliasTest.java | 138 +++++++++++++++++++++
.../server/http/CoordinatorDynamicConfigTest.java | 46 ++++++-
10 files changed, 321 insertions(+), 13 deletions(-)
diff --git a/docs/configuration/index.md b/docs/configuration/index.md
index aad78964f19..f5baf4d1375 100644
--- a/docs/configuration/index.md
+++ b/docs/configuration/index.md
@@ -793,7 +793,7 @@ The following table shows the dynamic configuration
properties for the Coordinat
|`replicateAfterLoadTimeout`|Boolean flag for whether or not additional
replication is needed for segments that have failed to load due to the expiry
of `druid.coordinator.load.timeout`. If this is set to true, the Coordinator
will attempt to replicate the failed segment on a different historical server.
This helps improve the segment availability if there are a few slow Historicals
in the cluster. However, the slow Historical may still load the segment later
and the Coordinator may issu [...]
|`turboLoadingNodes`| Experimental. List of Historical servers to place in
turbo loading mode. These servers use a larger thread-pool to load segments
faster but at the cost of query performance. For servers specified in
`turboLoadingNodes`, `druid.coordinator.loadqueuepeon.http.batchSize` is
ignored and the coordinator uses the value of the respective
`numLoadingThreads` instead.<br/>Please use this config with caution. All
servers should eventually be removed from this list once the se [...]
|`cloneServers`| Experimental. Map from target Historical server to source
Historical server which should be cloned by the target. The target Historical
does not participate in regular segment assignment or balancing. Instead, the
Coordinator mirrors any segment assignment made to the source Historical onto
the target Historical, so that the target becomes an exact copy of the source.
Segments on the target Historical do not count towards replica counts either.
If the source disappears, [...]
-|`historicalTierAliases`|Map from a virtual tier name to the set of real
Historical tier names it expands to. When a load/drop rule references a virtual
alias tier, the Coordinator replaces it with its real tiers — each receiving
the full replica count independently. The alias key itself is never loaded to
directly. For example, `{"hot": ["hot_1", "hot_2"]}` causes a rule of `{"hot":
2}` to load 2 replicas on each of `hot_1` and `hot_2`; `hot` receives no direct
assignment. An alias valu [...]
+|`historicalTierAliases`|Map from a virtual tier name to the set of real
Historical tier names it expands to. When a load/drop rule references a virtual
alias tier, the Coordinator replaces it with its real tiers — each receiving
the full replica count independently. The alias key itself is never loaded to
directly. For example, `{"hot": ["hot_1", "hot_2"]}` causes a rule of `{"hot":
2}` to load 2 replicas on each of `hot_1` and `hot_2`; `hot` receives no direct
assignment. An alias valu [...]
##### Smart segment loading
diff --git a/docs/operations/metrics.md b/docs/operations/metrics.md
index 6f87b7f5c96..8d71c6044f8 100644
--- a/docs/operations/metrics.md
+++ b/docs/operations/metrics.md
@@ -439,10 +439,10 @@ These metrics are emitted by the Druid Coordinator in
every run of the correspon
|`segment/unavailable/count`|Number of unique segments left to load until all
used segments are available for queries.|`dataSource`|0|
|`segment/underReplicated/count`|Number of segments, including replicas, left
to load until all used segments are available for queries.|`tier`,
`dataSource`|0|
|`segment/availableDeepStorageOnly/count`|Number of unique segments that are
only available for querying directly from deep storage.|`dataSource`|Varies|
-|`tier/historical/count`|Number of available historical nodes in each
tier.|`tier`|Varies|
-|`tier/replication/factor`|Configured maximum replication factor in each
tier.|`tier`|Varies|
-|`tier/required/capacity`|Total capacity in bytes required in each
tier.|`tier`|Varies|
-|`tier/total/capacity`|Total capacity in bytes available in each
tier.|`tier`|Varies|
+|`tier/historical/count`|Number of available historical nodes in each tier.
The `tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration), and
can be used to aggregate metrics across the tiers in an alias.|`tier`,
`tierAlias`|Varies|
+|`tier/replication/factor`|Configured maximum replication factor in each tier.
The `tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
+|`tier/required/capacity`|Total capacity in bytes required in each tier. The
`tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
+|`tier/total/capacity`|Total capacity in bytes available in each tier. The
`tierAlias` dimension is emitted only when the tier belongs to an alias
configured via
[`historicalTierAliases`](../configuration/index.md#dynamic-configuration).|`tier`,
`tierAlias`|Varies|
|`compact/task/count`|Number of tasks issued in the auto compaction run.|
|Varies|
|`compactTask/maxSlot/count`|Maximum number of task slots available for auto
compaction tasks in the auto compaction run.| |Varies|
|`compactTask/availableSlot/count`|Number of currently vacant task slots out
of the total slots allocated for auto compaction tasks. This value is computed
as the difference between the total number of task slots allocated for auto
compaction and the estimated number of task slots currently occupied by running
compaction tasks. The number of sub-tasks of each compaction task is estimated
to be `maxNumConcurrentSubTasks`.| |Varies|
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/CoordinatorDynamicConfig.java
b/server/src/main/java/org/apache/druid/server/coordinator/CoordinatorDynamicConfig.java
index 66652961e36..2c83e4a2c4a 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/CoordinatorDynamicConfig.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/CoordinatorDynamicConfig.java
@@ -37,6 +37,7 @@ import javax.annotation.Nullable;
import javax.validation.constraints.NotNull;
import java.util.Collection;
import java.util.EnumMap;
+import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Objects;
@@ -84,6 +85,13 @@ public class CoordinatorDynamicConfig
*/
private final Map<String, Set<String>> historicalTierAliases;
+ /**
+ * Reverse view of {@link #historicalTierAliases}: maps each physical tier
name
+ * to the alias tier it belongs to. Derived in the constructor and used to
tag
+ * coordinator metrics with their {@link Dimension#TIER_ALIAS}.
+ */
+ private final Map<String, String> tierToAliasName;
+
/**
* Stale pending segments belonging to the data sources in this list are not
killed by {@code
* KillStalePendingSegments}. In other words, segments in these data sources
are "protected".
@@ -184,6 +192,7 @@ public class CoordinatorDynamicConfig
this.historicalTierAliases = Configs.valueOrDefault(historicalTierAliases,
Map.of());
final Set<String> aliasKeys = this.historicalTierAliases.keySet();
+ final Set<String> seenTiers = new HashSet<>();
for (Set<String> mappedTiers : this.historicalTierAliases.values()) {
if (!Sets.intersection(mappedTiers, aliasKeys).isEmpty()) {
throw InvalidInput.exception(
@@ -191,7 +200,32 @@ public class CoordinatorDynamicConfig
this.historicalTierAliases
);
}
+ final Set<String> duplicateTiers = Sets.intersection(mappedTiers,
seenTiers);
+ if (!duplicateTiers.isEmpty()) {
+ throw InvalidInput.exception(
+ "historicalTierAliases [%s] is invalid. Physical tier(s) %s cannot
belong to more than one alias.",
+ this.historicalTierAliases,
+ duplicateTiers
+ );
+ }
+ seenTiers.addAll(mappedTiers);
+ }
+ this.tierToAliasName = computeTierToAliasName(this.historicalTierAliases);
+ }
+
+ /**
+ * Builds a reverse lookup from physical tier name to its alias tier name.
Each
+ * physical tier belongs to at most one alias (enforced during validation),
so
+ * the mapping is unambiguous.
+ */
+ private static Map<String, String> computeTierToAliasName(Map<String,
Set<String>> aliases)
+ {
+ if (aliases.isEmpty()) {
+ return Map.of();
}
+ final Map<String, String> reverse = new HashMap<>();
+ aliases.forEach((alias, tiers) -> tiers.forEach(tier -> reverse.put(tier,
alias)));
+ return reverse;
}
private Map<Dimension, String> validateDebugDimensions(Map<String, String>
debugDimensions)
@@ -364,6 +398,17 @@ public class CoordinatorDynamicConfig
return historicalTierAliases;
}
+ /**
+ * Reverse view of {@link #getHistoricalTierAliases()} mapping each physical
tier
+ * to the alias tier it belongs to. Used to tag coordinator metrics with
+ * {@link Dimension#TIER_ALIAS}. Not serialized; derived from {@code
historicalTierAliases}.
+ */
+ @JsonIgnore
+ public Map<String, String> getTierToAliasName()
+ {
+ return tierToAliasName;
+ }
+
/**
* List of servers to put in turbo-loading mode. These servers will use a
larger thread pool to load
* segments. This causes decreases the average time taken to load segments.
However, this also means less resources
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/duty/PrepareBalancerAndLoadQueues.java
b/server/src/main/java/org/apache/druid/server/coordinator/duty/PrepareBalancerAndLoadQueues.java
index f2029023a76..ec0ddcaf0d0 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/duty/PrepareBalancerAndLoadQueues.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/duty/PrepareBalancerAndLoadQueues.java
@@ -40,6 +40,7 @@ import org.apache.druid.timeline.DataSegment;
import java.util.HashSet;
import java.util.List;
+import java.util.Map;
import java.util.Set;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.Collectors;
@@ -92,7 +93,7 @@ public class PrepareBalancerAndLoadQueues implements
CoordinatorDuty
cancelLoadsOnDecommissioningServers(cluster);
final CoordinatorRunStats stats = params.getCoordinatorStats();
- collectHistoricalStats(cluster, stats);
+ collectHistoricalStats(cluster, stats, dynamicConfig.getTierToAliasName());
collectUsedSegmentStats(params, stats);
collectDebugStats(segmentLoadingConfig, stats);
@@ -170,10 +171,17 @@ public class PrepareBalancerAndLoadQueues implements
CoordinatorDuty
return cluster.build();
}
- private void collectHistoricalStats(DruidCluster cluster,
CoordinatorRunStats stats)
+ private void collectHistoricalStats(
+ DruidCluster cluster,
+ CoordinatorRunStats stats,
+ Map<String, String> tierToAliasName
+ )
{
cluster.getHistoricals().forEach((tier, historicals) -> {
- RowKey rowKey = RowKey.of(Dimension.TIER, tier);
+ final String alias = tierToAliasName.get(tier);
+ final RowKey rowKey = alias == null
+ ? RowKey.of(Dimension.TIER, tier)
+ : RowKey.with(Dimension.TIER,
tier).and(Dimension.TIER_ALIAS, alias);
stats.add(Stats.Tier.HISTORICAL_COUNT, rowKey, historicals.size());
long totalCapacity = 0;
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/loading/SegmentLoadingConfig.java
b/server/src/main/java/org/apache/druid/server/coordinator/loading/SegmentLoadingConfig.java
index d89c009f472..44f842ddcac 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/loading/SegmentLoadingConfig.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/loading/SegmentLoadingConfig.java
@@ -42,6 +42,7 @@ public class SegmentLoadingConfig
private final boolean useRoundRobinSegmentAssignment;
private final Map<String, Set<String>> historicalTierAliases;
+ private final Map<String, String> tierToAliasName;
/**
* Creates a new SegmentLoadingConfig with recomputed coordinator config
values
@@ -67,7 +68,8 @@ public class SegmentLoadingConfig
60,
true,
numBalancerThreads,
- dynamicConfig.getHistoricalTierAliases()
+ dynamicConfig.getHistoricalTierAliases(),
+ dynamicConfig.getTierToAliasName()
);
} else {
// Use the configured values
@@ -77,7 +79,8 @@ public class SegmentLoadingConfig
dynamicConfig.getReplicantLifetime(),
dynamicConfig.isUseRoundRobinSegmentAssignment(),
dynamicConfig.getBalancerComputeThreads(),
- dynamicConfig.getHistoricalTierAliases()
+ dynamicConfig.getHistoricalTierAliases(),
+ dynamicConfig.getTierToAliasName()
);
}
}
@@ -88,7 +91,8 @@ public class SegmentLoadingConfig
int maxLifetimeInLoadQueue,
boolean useRoundRobinSegmentAssignment,
int balancerComputeThreads,
- Map<String, Set<String>> historicalTierAliases
+ Map<String, Set<String>> historicalTierAliases,
+ Map<String, String> tierToAliasName
)
{
this.maxSegmentsInLoadQueue = maxSegmentsInLoadQueue;
@@ -97,6 +101,7 @@ public class SegmentLoadingConfig
this.useRoundRobinSegmentAssignment = useRoundRobinSegmentAssignment;
this.balancerComputeThreads = balancerComputeThreads;
this.historicalTierAliases = historicalTierAliases;
+ this.tierToAliasName = tierToAliasName;
}
public int getMaxSegmentsInLoadQueue()
@@ -128,4 +133,13 @@ public class SegmentLoadingConfig
{
return historicalTierAliases;
}
+
+ /**
+ * Maps each physical tier to the alias tier it belongs to. Used to tag
+ * coordinator metrics with {@link
org.apache.druid.server.coordinator.stats.Dimension#TIER_ALIAS}.
+ */
+ public Map<String, String> getTierToAliasName()
+ {
+ return tierToAliasName;
+ }
}
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
b/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
index 9aee83e92f2..fc6936e4df1 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
@@ -66,6 +66,7 @@ public class StrategicSegmentAssigner implements
SegmentActionHandler
private final boolean useRoundRobinAssignment;
private final Map<String, Set<String>> historicalTierAliases;
+ private final Map<String, String> tierToAliasName;
private final Map<String, Set<String>> datasourceToInvalidLoadTiers = new
HashMap<>();
private final Map<String, Integer> tierToHistoricalCount = new HashMap<>();
@@ -90,6 +91,7 @@ public class StrategicSegmentAssigner implements
SegmentActionHandler
this.useRoundRobinAssignment =
loadingConfig.isUseRoundRobinSegmentAssignment();
this.serverSelector = useRoundRobinAssignment ? new
RoundRobinServerSelector(cluster) : null;
this.historicalTierAliases = loadingConfig.getHistoricalTierAliases();
+ this.tierToAliasName = loadingConfig.getTierToAliasName();
cluster.getManagedHistoricals().forEach(
(tier, historicals) -> tierToHistoricalCount.put(tier,
historicals.size())
@@ -654,11 +656,25 @@ public class StrategicSegmentAssigner implements
SegmentActionHandler
private void reportTierCapacityStats(DataSegment segment, int
requiredReplicas, String tier)
{
- final RowKey rowKey = RowKey.of(Dimension.TIER, tier);
+ final RowKey rowKey = tierRowKey(tier);
stats.updateMax(Stats.Tier.REPLICATION_FACTOR, rowKey, requiredReplicas);
stats.add(Stats.Tier.REQUIRED_CAPACITY, rowKey, segment.getSize() *
requiredReplicas);
}
+ /**
+ * Builds a {@link RowKey} for the given physical tier, additionally tagging
it
+ * with {@link Dimension#TIER_ALIAS} when the tier belongs to an alias. This
lets
+ * metrics for aliased tiers (e.g. blue/green pairs) be aggregated by alias.
+ */
+ private RowKey tierRowKey(String tier)
+ {
+ final String alias = tierToAliasName.get(tier);
+ if (alias == null) {
+ return RowKey.of(Dimension.TIER, tier);
+ }
+ return RowKey.with(Dimension.TIER, tier).and(Dimension.TIER_ALIAS, alias);
+ }
+
@Override
public void broadcastSegment(DataSegment segment)
{
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/stats/Dimension.java
b/server/src/main/java/org/apache/druid/server/coordinator/stats/Dimension.java
index 2871aab26d8..dd06e176126 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/stats/Dimension.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/stats/Dimension.java
@@ -25,6 +25,7 @@ package org.apache.druid.server.coordinator.stats;
public enum Dimension
{
TIER("tier"),
+ TIER_ALIAS("tierAlias"),
TASK_TYPE("taskType"),
DATASOURCE("dataSource"),
DUTY("duty"),
diff --git
a/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
b/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
index 329f24d420e..1dce14d1dda 100644
---
a/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
+++
b/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
@@ -43,6 +43,8 @@ import
org.apache.druid.server.coordinator.loading.SegmentLoadQueueManager;
import org.apache.druid.server.coordinator.loading.StrategicSegmentAssigner;
import org.apache.druid.server.coordinator.loading.TestLoadQueuePeon;
import org.apache.druid.server.coordinator.stats.CoordinatorRunStats;
+import org.apache.druid.server.coordinator.stats.Dimension;
+import org.apache.druid.server.coordinator.stats.RowKey;
import org.apache.druid.server.coordinator.stats.Stats;
import org.apache.druid.timeline.DataSegment;
import org.apache.druid.timeline.partition.NoneShardSpec;
@@ -557,6 +559,46 @@ public class LoadRuleTest
Assert.assertEquals(1L, stats.getSegmentStat(Stats.Segments.ASSIGNED,
Tier.T3, TestDataSource.WIKI));
}
+ /**
+ * Verifies that tier capacity metrics for tiers belonging to an alias are
tagged
+ * with the {@link Dimension#TIER_ALIAS} dimension, so they can be
aggregated by
+ * alias, while the physical tier dimension is still present.
+ */
+ @Test
+ public void testHistoricalTierAliasesTagsCapacityStatsWithAlias()
+ {
+ // T1 is the virtual alias key; T2 and T3 are the real tiers it expands to
+ final ServerHolder hot1Server = createServer(Tier.T2);
+ final ServerHolder hot2Server = createServer(Tier.T3);
+ DruidCluster cluster = DruidCluster
+ .builder()
+ .addTier(Tier.T2, hot1Server)
+ .addTier(Tier.T3, hot2Server)
+ .build();
+
+ final DataSegment segment = createDataSegment(TestDataSource.WIKI);
+ LoadRule rule = loadForever(ImmutableMap.of(Tier.T1, 1));
+ CoordinatorRunStats stats = runRuleAndGetStats(
+ rule,
+ segment,
+ makeCoordinatorRuntimeParams(
+ cluster,
+ ImmutableMap.of(Tier.T1, Set.of(Tier.T2, Tier.T3)),
+ segment
+ )
+ );
+
+ // Required capacity is reported against the physical tier AND tagged with
the alias
+ final RowKey t2WithAlias = RowKey.with(Dimension.TIER,
Tier.T2).and(Dimension.TIER_ALIAS, Tier.T1);
+ final RowKey t3WithAlias = RowKey.with(Dimension.TIER,
Tier.T3).and(Dimension.TIER_ALIAS, Tier.T1);
+ Assert.assertEquals(segment.getSize(),
stats.get(Stats.Tier.REQUIRED_CAPACITY, t2WithAlias));
+ Assert.assertEquals(segment.getSize(),
stats.get(Stats.Tier.REQUIRED_CAPACITY, t3WithAlias));
+
+ // The same stat without the alias dimension is a different row and must
be absent
+ Assert.assertEquals(0L, stats.get(Stats.Tier.REQUIRED_CAPACITY,
RowKey.of(Dimension.TIER, Tier.T2)));
+ Assert.assertEquals(0L, stats.get(Stats.Tier.REQUIRED_CAPACITY,
RowKey.of(Dimension.TIER, Tier.T3)));
+ }
+
/**
* Verifies that an alias key tier with no servers does not fire an
invalid-tier
* alert, but an alias value tier with no servers does.
diff --git
a/server/src/test/java/org/apache/druid/server/coordinator/simulate/HistoricalTierAliasTest.java
b/server/src/test/java/org/apache/druid/server/coordinator/simulate/HistoricalTierAliasTest.java
new file mode 100644
index 00000000000..3564719c6b0
--- /dev/null
+++
b/server/src/test/java/org/apache/druid/server/coordinator/simulate/HistoricalTierAliasTest.java
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.server.coordinator.simulate;
+
+import org.apache.druid.client.DruidServer;
+import org.apache.druid.segment.TestDataSource;
+import org.apache.druid.server.coordinator.CoordinatorDynamicConfig;
+import org.apache.druid.server.coordinator.stats.Dimension;
+import org.apache.druid.server.coordinator.stats.Stats;
+import org.junit.Test;
+
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Verifies that when historical tiers are grouped under an alias (e.g. a
blue/green
+ * pair), the coordinator tags the per-tier capacity metrics with the
+ * {@link Dimension#TIER_ALIAS} dimension so they can be aggregated by alias.
+ */
+public class HistoricalTierAliasTest extends CoordinatorSimulationBaseTest
+{
+ private static final long SIZE_1TB = 1_000_000;
+ private static final String ALIAS = "hot";
+
+ private DruidServer historicalT1;
+ private DruidServer historicalT2;
+
+ private final String datasource = TestDataSource.WIKI;
+
+ @Override
+ public void setUp()
+ {
+ // T1 and T2 are interchangeable physical tiers grouped under the alias
"hot"
+ historicalT1 = createHistorical(1, Tier.T1, SIZE_1TB);
+ historicalT2 = createHistorical(1, Tier.T2, SIZE_1TB);
+ }
+
+ @Test
+ public void testCapacityMetricsAreTaggedWithAlias()
+ {
+ final CoordinatorSimulation sim =
+ CoordinatorSimulation.builder()
+ .withSegments(Segments.WIKI_10X1D)
+ .withServers(historicalT1, historicalT2)
+ // Rule targets the virtual alias, which expands
to T1 and T2
+ .withRules(datasource, Load.on(ALIAS,
1).forever())
+ .withDynamicConfig(
+ CoordinatorDynamicConfig.builder()
+
.withHistoricalTierAliases(
+ Map.of(ALIAS,
Set.of(Tier.T1, Tier.T2))
+ )
+
.withSmartSegmentLoading(true)
+ .build()
+ )
+ .withImmediateSegmentLoading(true)
+ .build();
+
+ startSimulation(sim);
+ runCoordinatorCycle();
+
+ final long expectedCapacity = SIZE_1TB << 20;
+
+ // tier/total/capacity is emitted per physical tier AND tagged with the
alias
+ verifyValue(
+ Stats.Tier.TOTAL_CAPACITY.getMetricName(),
+ Map.of(Dimension.TIER.reportedName(), Tier.T1,
Dimension.TIER_ALIAS.reportedName(), ALIAS),
+ expectedCapacity
+ );
+ verifyValue(
+ Stats.Tier.TOTAL_CAPACITY.getMetricName(),
+ Map.of(Dimension.TIER.reportedName(), Tier.T2,
Dimension.TIER_ALIAS.reportedName(), ALIAS),
+ expectedCapacity
+ );
+
+ // tier/historical/count carries the alias too, so it can be summed across
the pair
+ verifyValue(
+ Stats.Tier.HISTORICAL_COUNT.getMetricName(),
+ Map.of(Dimension.TIER.reportedName(), Tier.T1,
Dimension.TIER_ALIAS.reportedName(), ALIAS),
+ 1L
+ );
+ verifyValue(
+ Stats.Tier.HISTORICAL_COUNT.getMetricName(),
+ Map.of(Dimension.TIER.reportedName(), Tier.T2,
Dimension.TIER_ALIAS.reportedName(), ALIAS),
+ 1L
+ );
+ }
+
+ @Test
+ public void testCapacityMetricsHaveNoAliasWhenNotConfigured()
+ {
+ final CoordinatorSimulation sim =
+ CoordinatorSimulation.builder()
+ .withSegments(Segments.WIKI_10X1D)
+ .withServers(historicalT1, historicalT2)
+ .withRules(datasource, Load.on(Tier.T1,
1).andOn(Tier.T2, 1).forever())
+ .withDynamicConfig(
+ CoordinatorDynamicConfig.builder()
+
.withSmartSegmentLoading(true)
+ .build()
+ )
+ .withImmediateSegmentLoading(true)
+ .build();
+
+ startSimulation(sim);
+ runCoordinatorCycle();
+
+ final long expectedCapacity = SIZE_1TB << 20;
+
+ // Without an alias configured, capacity is reported against the physical
tier only
+ verifyValue(
+ Stats.Tier.TOTAL_CAPACITY.getMetricName(),
+ filterByTier(Tier.T1),
+ expectedCapacity
+ );
+ verifyValue(
+ Stats.Tier.TOTAL_CAPACITY.getMetricName(),
+ filterByTier(Tier.T2),
+ expectedCapacity
+ );
+ }
+}
diff --git
a/server/src/test/java/org/apache/druid/server/http/CoordinatorDynamicConfigTest.java
b/server/src/test/java/org/apache/druid/server/http/CoordinatorDynamicConfigTest.java
index f56344ec12e..53dcd009dc6 100644
---
a/server/src/test/java/org/apache/druid/server/http/CoordinatorDynamicConfigTest.java
+++
b/server/src/test/java/org/apache/druid/server/http/CoordinatorDynamicConfigTest.java
@@ -698,11 +698,55 @@ public class CoordinatorDynamicConfigTest
Assert.assertTrue("Throws correct virtual tier alias message",
exception.getMessage().contains("A virtual tier alias cannot be a physical
tier."));
}
+ @Test
+ public void testHistoricalTierAliasesRejectsTierInMultipleAliases()
+ {
+ Map<String, Set<String>> aliases = Map.of(
+ "hot", Set.of("tier_1", "tier_2"),
+ "warm", Set.of("tier_2", "tier_3")
+ );
+
+ DruidException exception = Assert.assertThrows(
+ DruidException.class,
+ () ->
CoordinatorDynamicConfig.builder().withHistoricalTierAliases(aliases).build()
+ );
+ Assert.assertTrue(
+ "Throws correct multi-alias message",
+ exception.getMessage().contains("cannot belong to more than one alias")
+ );
+ Assert.assertTrue(
+ "Names the offending tier",
+ exception.getMessage().contains("tier_2")
+ );
+ }
+
+ @Test
+ public void testGetTierToAliasName()
+ {
+ Map<String, Set<String>> aliases = Map.of(
+ "hot", Set.of("hot_1", "hot_2"),
+ "cold", Set.of("cold_1")
+ );
+ CoordinatorDynamicConfig config = CoordinatorDynamicConfig.builder()
+
.withHistoricalTierAliases(aliases)
+ .build();
+
+ Map<String, String> expected = Map.of(
+ "hot_1", "hot",
+ "hot_2", "hot",
+ "cold_1", "cold"
+ );
+ Assert.assertEquals(expected, config.getTierToAliasName());
+
+ // No aliases configured -> empty reverse map
+ Assert.assertEquals(Map.of(),
CoordinatorDynamicConfig.builder().build().getTierToAliasName());
+ }
+
@Test
public void testEquals()
{
EqualsVerifier.forClass(CoordinatorDynamicConfig.class)
- .withIgnoredFields("validDebugDimensions")
+ .withIgnoredFields("validDebugDimensions", "tierToAliasName")
.usingGetClass()
.verify();
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
