This is an automated email from the ASF dual-hosted git repository.
tuglu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 681cbdee15d feat: historical tier aliasing support in coordinator
dynamic config (#19204)
681cbdee15d is described below
commit 681cbdee15d279acf976a4e851da3e7a03bbba81
Author: jtuglu1 <[email protected]>
AuthorDate: Tue Apr 14 13:32:05 2026 -0700
feat: historical tier aliasing support in coordinator dynamic config
(#19204)
Introduces historical tier aliases in the Coordinator dynamic config. This
allows usage of a virtual "alias" tier in retention rules, e.g. hot to map to N
other historical tiers. The alias tier is virtual (it itself doesn't store
segments), but instead acts as a single identifier for a group of replica
tiers. This reduces friction for operators (to not need to duplicate their
retention rules N times) as well as enabling use-cases like:
1. Rolling deployments that use historical tiering for query/data isolation
across versions (e.g. ensure new version of historical on hot_new and old
version of historical on hot_old load the same data).
2. Historical tiers to be mirrored across different regions/AZs for
fault-tolerance.
---
docs/api-reference/dynamic-configuration-api.md | 5 +-
docs/configuration/index.md | 1 +
.../coordinator/CoordinatorDynamicConfig.java | 52 ++++++++-
.../coordinator/loading/SegmentLoadingConfig.java | 20 +++-
.../loading/StrategicSegmentAssigner.java | 36 +++++-
.../server/coordinator/rules/LoadRuleTest.java | 127 +++++++++++++++++++++
.../server/http/CoordinatorDynamicConfigTest.java | 53 +++++++++
.../coordinator-dynamic-config-completions.ts | 5 +
.../coordinator-dynamic-config.tsx | 13 +++
9 files changed, 299 insertions(+), 13 deletions(-)
diff --git a/docs/api-reference/dynamic-configuration-api.md
b/docs/api-reference/dynamic-configuration-api.md
index 95166aa3c54..259e50c8800 100644
--- a/docs/api-reference/dynamic-configuration-api.md
+++ b/docs/api-reference/dynamic-configuration-api.md
@@ -107,7 +107,10 @@ Host: http://ROUTER_IP:ROUTER_PORT
"smartSegmentLoading": true,
"debugDimensions": null,
"turboLoadingNodes": [],
- "cloneServers": {}
+ "cloneServers": {},
+ "historicalTierAliases": {
+ "hot": ["hot_1", "hot_2"]
+ }
}
```
diff --git a/docs/configuration/index.md b/docs/configuration/index.md
index 172d8062a5e..e309126863a 100644
--- a/docs/configuration/index.md
+++ b/docs/configuration/index.md
@@ -835,6 +835,7 @@ The following table shows the dynamic configuration
properties for the Coordinat
|`replicateAfterLoadTimeout`|Boolean flag for whether or not additional
replication is needed for segments that have failed to load due to the expiry
of `druid.coordinator.load.timeout`. If this is set to true, the Coordinator
will attempt to replicate the failed segment on a different historical server.
This helps improve the segment availability if there are a few slow Historicals
in the cluster. However, the slow Historical may still load the segment later
and the Coordinator may issu [...]
|`turboLoadingNodes`| Experimental. List of Historical servers to place in
turbo loading mode. These servers use a larger thread-pool to load segments
faster but at the cost of query performance. For servers specified in
`turboLoadingNodes`, `druid.coordinator.loadqueuepeon.http.batchSize` is
ignored and the coordinator uses the value of the respective
`numLoadingThreads` instead.<br/>Please use this config with caution. All
servers should eventually be removed from this list once the se [...]
|`cloneServers`| Experimental. Map from target Historical server to source
Historical server which should be cloned by the target. The target Historical
does not participate in regular segment assignment or balancing. Instead, the
Coordinator mirrors any segment assignment made to the source Historical onto
the target Historical, so that the target becomes an exact copy of the source.
Segments on the target Historical do not count towards replica counts either.
If the source disappears, [...]
+|`historicalTierAliases`|Map from a virtual tier name to the set of real
Historical tier names it expands to. When a load/drop rule references a virtual
alias tier, the Coordinator replaces it with its real tiers — each receiving
the full replica count independently. The alias key itself is never loaded to
directly. For example, `{"hot": ["hot_1", "hot_2"]}` causes a rule of `{"hot":
2}` to load 2 replicas on each of `hot_1` and `hot_2`; `hot` receives no direct
assignment. An alias valu [...]
##### Smart segment loading
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/CoordinatorDynamicConfig.java
b/server/src/main/java/org/apache/druid/server/coordinator/CoordinatorDynamicConfig.java
index f34c3b8ae39..66652961e36 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/CoordinatorDynamicConfig.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/CoordinatorDynamicConfig.java
@@ -23,6 +23,7 @@ import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Sets;
import org.apache.druid.common.config.Configs;
import org.apache.druid.common.config.JacksonConfigManager;
import org.apache.druid.error.InvalidInput;
@@ -76,6 +77,13 @@ public class CoordinatorDynamicConfig
private final Set<String> turboLoadingNodes;
private final Map<String, String> cloneServers;
+ /**
+ * Map from alias tier name to the set of actual tier names it represents.
+ * For example, "hot" -> {"hot_1", "hot_2"} allows rules targeting "hot" to
apply
+ * to servers in either tier.
+ */
+ private final Map<String, Set<String>> historicalTierAliases;
+
/**
* Stale pending segments belonging to the data sources in this list are not
killed by {@code
* KillStalePendingSegments}. In other words, segments in these data sources
are "protected".
@@ -126,7 +134,8 @@ public class CoordinatorDynamicConfig
@JsonProperty("smartSegmentLoading") @Nullable Boolean
smartSegmentLoading,
@JsonProperty("debugDimensions") @Nullable Map<String, String>
debugDimensions,
@JsonProperty("turboLoadingNodes") @Nullable Set<String>
turboLoadingNodes,
- @JsonProperty("cloneServers") @Nullable Map<String, String> cloneServers
+ @JsonProperty("cloneServers") @Nullable Map<String, String> cloneServers,
+ @JsonProperty("historicalTierAliases") @Nullable Map<String,
Set<String>> historicalTierAliases
)
{
this.markSegmentAsUnusedDelayMillis =
@@ -172,6 +181,17 @@ public class CoordinatorDynamicConfig
this.validDebugDimensions = validateDebugDimensions(debugDimensions);
this.turboLoadingNodes = Configs.valueOrDefault(turboLoadingNodes,
Set.of());
this.cloneServers = Configs.valueOrDefault(cloneServers, Map.of());
+
+ this.historicalTierAliases = Configs.valueOrDefault(historicalTierAliases,
Map.of());
+ final Set<String> aliasKeys = this.historicalTierAliases.keySet();
+ for (Set<String> mappedTiers : this.historicalTierAliases.values()) {
+ if (!Sets.intersection(mappedTiers, aliasKeys).isEmpty()) {
+ throw InvalidInput.exception(
+ "historicalTierAliases [%s] is invalid. A virtual tier alias
cannot be a physical tier.",
+ this.historicalTierAliases
+ );
+ }
+ }
}
private Map<Dimension, String> validateDebugDimensions(Map<String, String>
debugDimensions)
@@ -338,6 +358,12 @@ public class CoordinatorDynamicConfig
return cloneServers;
}
+ @JsonProperty
+ public Map<String, Set<String>> getHistoricalTierAliases()
+ {
+ return historicalTierAliases;
+ }
+
/**
* List of servers to put in turbo-loading mode. These servers will use a
larger thread pool to load
* segments. This causes decreases the average time taken to load segments.
However, this also means less resources
@@ -371,6 +397,7 @@ public class CoordinatorDynamicConfig
", replicateAfterLoadTimeout=" + replicateAfterLoadTimeout +
", turboLoadingNodes=" + turboLoadingNodes +
", cloneServers=" + cloneServers +
+ ", historicalTierAliases=" + historicalTierAliases +
'}';
}
@@ -407,7 +434,8 @@ public class CoordinatorDynamicConfig
&& Objects.equals(decommissioningNodes, that.decommissioningNodes)
&& Objects.equals(turboLoadingNodes, that.turboLoadingNodes)
&& Objects.equals(debugDimensions, that.debugDimensions)
- && Objects.equals(cloneServers, that.cloneServers);
+ && Objects.equals(cloneServers, that.cloneServers)
+ && Objects.equals(historicalTierAliases,
that.historicalTierAliases);
}
@Override
@@ -431,7 +459,8 @@ public class CoordinatorDynamicConfig
pauseCoordination,
debugDimensions,
turboLoadingNodes,
- cloneServers
+ cloneServers,
+ historicalTierAliases
);
}
@@ -488,6 +517,7 @@ public class CoordinatorDynamicConfig
private Boolean smartSegmentLoading;
private Set<String> turboLoadingNodes;
private Map<String, String> cloneServers;
+ private Map<String, Set<String>> historicalTierAliases;
public Builder()
{
@@ -512,7 +542,8 @@ public class CoordinatorDynamicConfig
@JsonProperty("smartSegmentLoading") @Nullable Boolean
smartSegmentLoading,
@JsonProperty("debugDimensions") @Nullable Map<String, String>
debugDimensions,
@JsonProperty("turboLoadingNodes") @Nullable Set<String>
turboLoadingNodes,
- @JsonProperty("cloneServers") @Nullable Map<String, String>
cloneServers
+ @JsonProperty("cloneServers") @Nullable Map<String, String>
cloneServers,
+ @JsonProperty("historicalTierAliases") @Nullable Map<String,
Set<String>> historicalTierAliases
)
{
this.markSegmentAsUnusedDelayMillis = markSegmentAsUnusedDelayMillis;
@@ -533,6 +564,7 @@ public class CoordinatorDynamicConfig
this.debugDimensions = debugDimensions;
this.turboLoadingNodes = turboLoadingNodes;
this.cloneServers = cloneServers;
+ this.historicalTierAliases = historicalTierAliases;
}
public Builder withMarkSegmentAsUnusedDelayMillis(long leadingTimeMillis)
@@ -631,6 +663,12 @@ public class CoordinatorDynamicConfig
return this;
}
+ public Builder withHistoricalTierAliases(Map<String, Set<String>>
historicalTierAliases)
+ {
+ this.historicalTierAliases = historicalTierAliases;
+ return this;
+ }
+
/**
* Builds a CoordinatoryDynamicConfig using either the configured values,
or
* the default value if not configured.
@@ -658,7 +696,8 @@ public class CoordinatorDynamicConfig
valueOrDefault(smartSegmentLoading, Defaults.SMART_SEGMENT_LOADING),
debugDimensions,
turboLoadingNodes,
- cloneServers
+ cloneServers,
+ historicalTierAliases
);
}
@@ -690,7 +729,8 @@ public class CoordinatorDynamicConfig
valueOrDefault(smartSegmentLoading,
defaults.isSmartSegmentLoading()),
valueOrDefault(debugDimensions, defaults.getDebugDimensions()),
valueOrDefault(turboLoadingNodes, defaults.getTurboLoadingNodes()),
- valueOrDefault(cloneServers, defaults.getCloneServers())
+ valueOrDefault(cloneServers, defaults.getCloneServers()),
+ valueOrDefault(historicalTierAliases,
defaults.getHistoricalTierAliases())
);
}
}
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/loading/SegmentLoadingConfig.java
b/server/src/main/java/org/apache/druid/server/coordinator/loading/SegmentLoadingConfig.java
index 26fcd220c25..d89c009f472 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/loading/SegmentLoadingConfig.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/loading/SegmentLoadingConfig.java
@@ -22,6 +22,9 @@ package org.apache.druid.server.coordinator.loading;
import org.apache.druid.java.util.common.logger.Logger;
import org.apache.druid.server.coordinator.CoordinatorDynamicConfig;
+import java.util.Map;
+import java.util.Set;
+
/**
* Contains recomputed configs from {@link CoordinatorDynamicConfig} based on
* whether {@link CoordinatorDynamicConfig#isSmartSegmentLoading} is enabled
or not.
@@ -38,6 +41,8 @@ public class SegmentLoadingConfig
private final boolean useRoundRobinSegmentAssignment;
+ private final Map<String, Set<String>> historicalTierAliases;
+
/**
* Creates a new SegmentLoadingConfig with recomputed coordinator config
values
* based on whether {@link CoordinatorDynamicConfig#isSmartSegmentLoading()}
@@ -61,7 +66,8 @@ public class SegmentLoadingConfig
replicationThrottleLimit,
60,
true,
- numBalancerThreads
+ numBalancerThreads,
+ dynamicConfig.getHistoricalTierAliases()
);
} else {
// Use the configured values
@@ -70,7 +76,8 @@ public class SegmentLoadingConfig
dynamicConfig.getReplicationThrottleLimit(),
dynamicConfig.getReplicantLifetime(),
dynamicConfig.isUseRoundRobinSegmentAssignment(),
- dynamicConfig.getBalancerComputeThreads()
+ dynamicConfig.getBalancerComputeThreads(),
+ dynamicConfig.getHistoricalTierAliases()
);
}
}
@@ -80,7 +87,8 @@ public class SegmentLoadingConfig
int replicationThrottleLimit,
int maxLifetimeInLoadQueue,
boolean useRoundRobinSegmentAssignment,
- int balancerComputeThreads
+ int balancerComputeThreads,
+ Map<String, Set<String>> historicalTierAliases
)
{
this.maxSegmentsInLoadQueue = maxSegmentsInLoadQueue;
@@ -88,6 +96,7 @@ public class SegmentLoadingConfig
this.maxLifetimeInLoadQueue = maxLifetimeInLoadQueue;
this.useRoundRobinSegmentAssignment = useRoundRobinSegmentAssignment;
this.balancerComputeThreads = balancerComputeThreads;
+ this.historicalTierAliases = historicalTierAliases;
}
public int getMaxSegmentsInLoadQueue()
@@ -114,4 +123,9 @@ public class SegmentLoadingConfig
{
return balancerComputeThreads;
}
+
+ public Map<String, Set<String>> getHistoricalTierAliases()
+ {
+ return historicalTierAliases;
+ }
}
diff --git
a/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
b/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
index 2b7bbdb49e7..d4fbfd82e31 100644
---
a/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
+++
b/server/src/main/java/org/apache/druid/server/coordinator/loading/StrategicSegmentAssigner.java
@@ -64,6 +64,7 @@ public class StrategicSegmentAssigner implements
SegmentActionHandler
private final BalancerStrategy strategy;
private final boolean useRoundRobinAssignment;
+ private final Map<String, Set<String>> historicalTierAliases;
private final Map<String, Set<String>> datasourceToInvalidLoadTiers = new
HashMap<>();
private final Map<String, Integer> tierToHistoricalCount = new HashMap<>();
@@ -87,6 +88,7 @@ public class StrategicSegmentAssigner implements
SegmentActionHandler
this.replicationThrottler = createReplicationThrottler(cluster,
loadingConfig);
this.useRoundRobinAssignment =
loadingConfig.isUseRoundRobinSegmentAssignment();
this.serverSelector = useRoundRobinAssignment ? new
RoundRobinServerSelector(cluster) : null;
+ this.historicalTierAliases = loadingConfig.getHistoricalTierAliases();
cluster.getManagedHistoricals().forEach(
(tier, historicals) -> tierToHistoricalCount.put(tier,
historicals.size())
@@ -198,17 +200,45 @@ public class StrategicSegmentAssigner implements
SegmentActionHandler
}
}
+ /**
+ * Resolves alias tiers in the given tier-to-replica-count map. For each tier
+ * that is a key in {@link #historicalTierAliases}, the entry is replaced by
+ * one entry per alias value — each receiving the same replica count. The
alias
+ * key itself is treated as a virtual tier and is not kept in the result.
Tiers
+ * not present in {@link #historicalTierAliases} are passed through
unchanged.
+ * Explicit counts already present in the map are not overwritten by alias
expansion.
+ */
+ private Map<String, Integer> expandWithAliases(Map<String, Integer>
tierToReplicaCount)
+ {
+ if (historicalTierAliases.isEmpty()) {
+ return tierToReplicaCount;
+ }
+
+ final Map<String, Integer> expanded = new HashMap<>();
+ tierToReplicaCount.forEach((tier, replicaCount) -> {
+ final Set<String> aliases = historicalTierAliases.get(tier);
+ if (aliases != null) {
+ // tier is a virtual alias key — replace it with its real tiers
+ aliases.forEach(alias -> expanded.putIfAbsent(alias, replicaCount));
+ } else {
+ expanded.put(tier, replicaCount);
+ }
+ });
+ return expanded;
+ }
+
@Override
public void replicateSegment(DataSegment segment, Map<String, Integer>
tierToReplicaCount)
{
+ final Map<String, Integer> effectiveTierToReplicaCount =
expandWithAliases(tierToReplicaCount);
final Set<String> allTiersInCluster =
Sets.newHashSet(cluster.getTierNames());
- if (tierToReplicaCount.isEmpty()) {
+ if (effectiveTierToReplicaCount.isEmpty()) {
// Track the counts for a segment even if it requires 0 replicas on all
tiers
replicaCountMap.computeIfAbsent(segment.getId(),
DruidServer.DEFAULT_TIER);
} else {
// Identify empty tiers and determine total required replicas
- tierToReplicaCount.forEach((tier, requiredReplicas) -> {
+ effectiveTierToReplicaCount.forEach((tier, requiredReplicas) -> {
reportTierCapacityStats(segment, requiredReplicas, tier);
SegmentReplicaCount replicaCount =
replicaCountMap.computeIfAbsent(segment.getId(), tier);
@@ -237,7 +267,7 @@ public class StrategicSegmentAssigner implements
SegmentActionHandler
dropsQueued += updateReplicasInTier(
segment,
tier,
- tierToReplicaCount.getOrDefault(tier, 0),
+ effectiveTierToReplicaCount.getOrDefault(tier, 0),
replicaSurplus - dropsQueued
);
}
diff --git
a/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
b/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
index 5cd5dd9d0f9..329f24d420e 100644
---
a/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
+++
b/server/src/test/java/org/apache/druid/server/coordinator/rules/LoadRuleTest.java
@@ -58,6 +58,7 @@ import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
+import java.util.Set;
import java.util.concurrent.atomic.AtomicInteger;
/**
@@ -523,10 +524,136 @@ public class LoadRuleTest
);
}
+ /**
+ * Verifies that a load rule targeting a virtual alias tier is applied only
to
+ * the real tiers in the alias set — the alias key itself receives no
assignment.
+ */
+ @Test
+ public void testHistoricalTierAliasesAppliesOnlyToAliasTiers()
+ {
+ // T1 is the virtual alias key and has no servers; T2 and T3 are the real
tiers
+ final ServerHolder hot1Server = createServer(Tier.T2);
+ final ServerHolder hot2Server = createServer(Tier.T3);
+ DruidCluster cluster = DruidCluster
+ .builder()
+ .addTier(Tier.T2, hot1Server)
+ .addTier(Tier.T3, hot2Server)
+ .build();
+
+ final DataSegment segment = createDataSegment(TestDataSource.WIKI);
+ LoadRule rule = loadForever(ImmutableMap.of(Tier.T1, 1));
+ CoordinatorRunStats stats = runRuleAndGetStats(
+ rule,
+ segment,
+ makeCoordinatorRuntimeParams(
+ cluster,
+ ImmutableMap.of(Tier.T1, Set.of(Tier.T2, Tier.T3)),
+ segment
+ )
+ );
+
+ Assert.assertEquals(0L, stats.getSegmentStat(Stats.Segments.ASSIGNED,
Tier.T1, TestDataSource.WIKI));
+ Assert.assertEquals(1L, stats.getSegmentStat(Stats.Segments.ASSIGNED,
Tier.T2, TestDataSource.WIKI));
+ Assert.assertEquals(1L, stats.getSegmentStat(Stats.Segments.ASSIGNED,
Tier.T3, TestDataSource.WIKI));
+ }
+
+ /**
+ * Verifies that an alias key tier with no servers does not fire an
invalid-tier
+ * alert, but an alias value tier with no servers does.
+ */
+ @Test
+ public void testHistoricalTierAliasesInvalidTierAlerts()
+ {
+ // Only T2 and T3 have servers; T1 ("hot") is a pure alias key with no
servers
+ final ServerHolder t2Server = createServer(Tier.T2);
+ DruidCluster cluster = DruidCluster
+ .builder()
+ .addTier(Tier.T2, t2Server)
+ .build();
+
+ final DataSegment segment = createDataSegment(TestDataSource.WIKI);
+ LoadRule rule = loadForever(ImmutableMap.of(Tier.T1, 1));
+
+ // T1 is an alias key -> no invalid-tier alert for T1.
+ // T3 is an alias value with no servers -> invalid-tier alert fires for T3.
+ DruidCoordinatorRuntimeParams params = makeCoordinatorRuntimeParams(
+ cluster,
+ ImmutableMap.of(Tier.T1, Set.of(Tier.T2, Tier.T3)),
+ segment
+ );
+ rule.run(segment, params.getSegmentAssigner());
+ Map<String, Set<String>> invalidTiers =
params.getSegmentAssigner().getDatasourceToInvalidLoadTiers();
+
+ Assert.assertFalse(
+ "Alias key tier should not trigger an invalid-tier alert",
+ invalidTiers.getOrDefault(TestDataSource.WIKI,
Collections.emptySet()).contains(Tier.T1)
+ );
+ Assert.assertTrue(
+ "Alias value tier with no servers should trigger an invalid-tier
alert",
+ invalidTiers.getOrDefault(TestDataSource.WIKI,
Collections.emptySet()).contains(Tier.T3)
+ );
+ }
+
+ /**
+ * Verifies that an explicit replica count for an alias value tier in the
rule
+ * is not overwritten by the alias expansion, and the virtual alias key tier
+ * itself receives no assignment.
+ */
+ @Test
+ public void testHistoricalTierAliasesDoesNotOverwriteExplicitCount()
+ {
+ final ServerHolder t2Server1 = createServer(Tier.T2);
+ final ServerHolder t2Server2 = createServer(Tier.T2);
+ DruidCluster cluster = DruidCluster
+ .builder()
+ .addTier(Tier.T2, t2Server1, t2Server2)
+ .build();
+
+ final DataSegment segment = createDataSegment(TestDataSource.WIKI);
+ // Rule explicitly sets T2 to 2; T1 is a virtual alias for T2 with count 1.
+ // T2's explicit count of 2 must win over the alias-derived count of 1.
+ LoadRule rule = loadForever(ImmutableMap.of(Tier.T1, 1, Tier.T2, 2));
+ CoordinatorRunStats stats = runRuleAndGetStats(
+ rule,
+ segment,
+ makeCoordinatorRuntimeParams(
+ cluster,
+ ImmutableMap.of(Tier.T1, Set.of(Tier.T2)),
+ segment
+ )
+ );
+
+ Assert.assertEquals(0L, stats.getSegmentStat(Stats.Segments.ASSIGNED,
Tier.T1, TestDataSource.WIKI));
+ Assert.assertEquals(2L, stats.getSegmentStat(Stats.Segments.ASSIGNED,
Tier.T2, TestDataSource.WIKI));
+ }
+
+ private DruidCoordinatorRuntimeParams makeCoordinatorRuntimeParams(
+ DruidCluster druidCluster,
+ Map<String, Set<String>> historicalTierAliases,
+ DataSegment... usedSegments
+ )
+ {
+ return DruidCoordinatorRuntimeParams
+ .builder()
+ .withDruidCluster(druidCluster)
+ .withBalancerStrategy(balancerStrategy)
+ .withUsedSegments(usedSegments)
+ .withDynamicConfigs(
+ CoordinatorDynamicConfig.builder()
+ .withSmartSegmentLoading(false)
+
.withUseRoundRobinSegmentAssignment(useRoundRobinAssignment)
+
.withHistoricalTierAliases(historicalTierAliases)
+ .build()
+ )
+ .withSegmentAssignerUsing(loadQueueManager)
+ .build();
+ }
+
private static class Tier
{
static final String T1 = "tier1";
static final String T2 = "tier2";
+ static final String T3 = "tier3";
}
@Test
diff --git
a/server/src/test/java/org/apache/druid/server/http/CoordinatorDynamicConfigTest.java
b/server/src/test/java/org/apache/druid/server/http/CoordinatorDynamicConfigTest.java
index 22f3ccbdd4d..f56344ec12e 100644
---
a/server/src/test/java/org/apache/druid/server/http/CoordinatorDynamicConfigTest.java
+++
b/server/src/test/java/org/apache/druid/server/http/CoordinatorDynamicConfigTest.java
@@ -23,6 +23,7 @@ import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import nl.jqno.equalsverifier.EqualsVerifier;
+import org.apache.druid.error.DruidException;
import org.apache.druid.segment.TestHelper;
import org.apache.druid.server.coordinator.CoordinatorDynamicConfig;
import org.apache.druid.utils.JvmUtils;
@@ -279,6 +280,7 @@ public class CoordinatorDynamicConfigTest
false,
null,
ImmutableSet.of("host1"),
+ null,
null
);
Assert.assertTrue(config.getSpecificDataSourcesToKillUnusedSegmentsIn().isEmpty());
@@ -305,6 +307,7 @@ public class CoordinatorDynamicConfigTest
false,
null,
ImmutableSet.of("host1"),
+ null,
null
);
Assert.assertEquals(ImmutableSet.of("test1"),
config.getSpecificDataSourcesToKillUnusedSegmentsIn());
@@ -645,6 +648,56 @@ public class CoordinatorDynamicConfigTest
return Math.max(1, JvmUtils.getRuntimeInfo().getAvailableProcessors() / 2);
}
+ @Test
+ public void testHistoricalTierAliases() throws Exception
+ {
+ // Basic set and get via builder
+ Map<String, Set<String>> aliases = Map.of(
+ "hot", Set.of("hot_1", "hot_2"),
+ "cold", Set.of("cold_1")
+ );
+ CoordinatorDynamicConfig config = CoordinatorDynamicConfig.builder()
+
.withHistoricalTierAliases(aliases)
+ .build();
+ Assert.assertEquals(aliases, config.getHistoricalTierAliases());
+
+ // build(defaults) propagates aliases when not overridden
+ CoordinatorDynamicConfig updated =
CoordinatorDynamicConfig.builder().build(config);
+ Assert.assertEquals(aliases, updated.getHistoricalTierAliases());
+
+ // Serde roundtrip with duplicate values in the JSON array — duplicates
must be deduplicated
+ String jsonWithDupes = "{"
+ + "\"historicalTierAliases\": {"
+ + " \"hot\": [\"hot_1\", \"hot_2\", \"hot_1\"]"
+ + "}"
+ + "}";
+ CoordinatorDynamicConfig deserialized = mapper.readValue(
+ mapper.writeValueAsString(mapper.readValue(jsonWithDupes,
CoordinatorDynamicConfig.class)),
+ CoordinatorDynamicConfig.class
+ );
+ Assert.assertEquals(Set.of("hot_1", "hot_2"),
deserialized.getHistoricalTierAliases().get("hot"));
+
+ // Absent field defaults to empty map
+ CoordinatorDynamicConfig defaultConfig =
CoordinatorDynamicConfig.builder().build();
+ Assert.assertEquals(Map.of(), defaultConfig.getHistoricalTierAliases());
+ }
+
+ @Test
+ public void testHistoricalTierAliasesNoCycle()
+ {
+ Map<String, Set<String>> aliases = Map.of(
+ "hot", Set.of("hot", "hot_2"),
+ "cold", Set.of("cold_1"),
+ "another", Set.of("hot")
+ );
+
+ DruidException exception = Assert.assertThrows(
+ DruidException.class,
+ () ->
CoordinatorDynamicConfig.builder().withHistoricalTierAliases(aliases).build()
+ );
+ Assert.assertTrue("Throws correct virtual tier alias message",
exception.getMessage().contains("A virtual tier alias cannot be a physical
tier."));
+ }
+
@Test
public void testEquals()
{
diff --git
a/web-console/src/dialogs/coordinator-dynamic-config-dialog/coordinator-dynamic-config-completions.ts
b/web-console/src/dialogs/coordinator-dynamic-config-dialog/coordinator-dynamic-config-completions.ts
index 7a5cc2b7ee8..2f0ca83ae0e 100644
---
a/web-console/src/dialogs/coordinator-dynamic-config-dialog/coordinator-dynamic-config-completions.ts
+++
b/web-console/src/dialogs/coordinator-dynamic-config-dialog/coordinator-dynamic-config-completions.ts
@@ -93,6 +93,11 @@ export const COORDINATOR_DYNAMIC_CONFIG_COMPLETIONS:
JsonCompletionRule[] = [
value: 'turboLoadingNodes',
documentation: 'List of Historical servers to place in turbo loading
mode (experimental)',
},
+ {
+ value: 'historicalTierAliases',
+ documentation:
+ 'Map of virtual alias tier names to sets of physical historical
tiers they represent',
+ },
],
},
// Properties only available when smartSegmentLoading is false
diff --git
a/web-console/src/druid-models/coordinator-dynamic-config/coordinator-dynamic-config.tsx
b/web-console/src/druid-models/coordinator-dynamic-config/coordinator-dynamic-config.tsx
index 3794ecb6037..4b7a84090ae 100644
---
a/web-console/src/druid-models/coordinator-dynamic-config/coordinator-dynamic-config.tsx
+++
b/web-console/src/druid-models/coordinator-dynamic-config/coordinator-dynamic-config.tsx
@@ -39,6 +39,7 @@ export interface CoordinatorDynamicConfig {
useRoundRobinSegmentAssignment?: boolean;
smartSegmentLoading?: boolean;
turboLoadingNodes?: string[];
+ historicalTierAliases?: Record<string, string[]>;
// Undocumented
debugDimensions?: any;
@@ -265,4 +266,16 @@ export const COORDINATOR_DYNAMIC_CONFIG_FIELDS:
Field<CoordinatorDynamicConfig>[
</>
),
},
+ {
+ name: 'historicalTierAliases',
+ type: 'json',
+ placeholder: '{"hot": ["hot_v1", "hot_v2"]}',
+ info: (
+ <>
+ A map of virtual alias tier names to the sets of physical historical
tiers they represent.
+ Retention rules can reference an alias tier instead of listing each
physical tier
+ separately. An alias tier cannot itself be a physical tier name.
+ </>
+ ),
+ },
];
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
