FrankChen021 commented on code in PR #19051:
URL: https://github.com/apache/druid/pull/19051#discussion_r3141586117
##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/CascadingReindexingTemplate.java:
##########
@@ -262,6 +288,121 @@ private static boolean intervalEndsAfter(Interval
interval, DateTime boundary)
return interval.getEnd().isAfter(boundary);
}
+ /**
+ * Generates a timeline view showing the search intervals and their
associated reindexing
+ * configurations. This is useful for operators to understand how rules are
applied across
+ * different time periods and to preview the effects of rule changes before
they are applied.
+ *
+ * @param referenceTime the reference time to use for computing rule periods
(typically DateTime.now())
+ * @return a view of the reindexing timeline with intervals and their configs
+ */
+ public ReindexingTimelineView getReindexingTimelineView(DateTime
referenceTime)
+ {
+ if (!ruleProvider.isReady()) {
+ LOG.info(
+ "Rule provider [%s] is not ready, returning empty timeline for
dataSource[%s]",
+ ruleProvider.getType(),
+ dataSource
+ );
+ return new ReindexingTimelineView(dataSource, referenceTime, null,
Collections.emptyList(), null);
+ }
+
+ List<IntervalPartitioningInfo> searchIntervals;
+ try {
+ searchIntervals = generateAlignedSearchIntervals(referenceTime);
+ }
+ catch (SegmentGranularityTimelineValidationException e) {
+ return createValidationErrorView(
+ e,
+ referenceTime,
+ "INVALID_GRANULARITY_TIMELINE",
+ e.getOlderInterval().toString(),
+ e.getOlderGranularity().toString(),
+ e.getNewerInterval().toString(),
+ e.getNewerGranularity().toString()
+ );
+ }
+ catch (IAE e) {
+ return createValidationErrorView(
+ e,
+ referenceTime,
+ "VALIDATION_ERROR",
+ null,
+ null,
+ null,
+ null
+ );
+ }
+
+ if (searchIntervals.isEmpty()) {
+ LOG.warn("No search intervals generated for dataSource[%s]", dataSource);
+ return new ReindexingTimelineView(dataSource, referenceTime, null,
Collections.emptyList(), null);
+ }
+
+ // Calculate effective end time based on skip offset
+ DateTime effectiveEndTime = referenceTime;
+ ReindexingTimelineView.SkipOffsetInfo skipOffsetInfo = null;
+
+ if (skipOffsetFromNow != null) {
+ effectiveEndTime = referenceTime.minus(skipOffsetFromNow);
+ skipOffsetInfo = new ReindexingTimelineView.SkipOffsetInfo(
+ "skipOffsetFromNow",
+ skipOffsetFromNow,
+ true,
+ effectiveEndTime,
+ null
+ );
+ } else if (skipOffsetFromLatest != null) {
+ // skipOffsetFromLatest requires actual timeline data, so we can't apply
it in preview mode
+ skipOffsetInfo = new ReindexingTimelineView.SkipOffsetInfo(
+ "skipOffsetFromLatest",
+ skipOffsetFromLatest,
+ false,
+ null,
+ "Requires actual segment timeline data"
+ );
+ }
+
+ // Build configs for each interval
+ List<ReindexingTimelineView.IntervalConfig> intervalConfigs = new
ArrayList<>();
+ for (IntervalPartitioningInfo intervalInfo : searchIntervals) {
+ Interval searchInterval = intervalInfo.getInterval();
+
+ // Check if interval extends past skip offset
+ if (intervalEndsAfter(searchInterval, effectiveEndTime)) {
+ // Include in timeline, but indicate skipped by applying no rules.
+ intervalConfigs.add(new ReindexingTimelineView.IntervalConfig(
+ searchInterval,
+ 0,
+ null,
+ Collections.emptyList()
+ ));
+ continue;
+ }
+
+ InlineSchemaDataSourceCompactionConfig.Builder builder =
createBaseBuilder();
+ ReindexingConfigBuilder configBuilder = new ReindexingConfigBuilder(
+ ruleProvider,
+ searchInterval,
+ referenceTime,
+ searchIntervals,
+ null
Review Comment:
P2 Timeline preview drops template tuningConfig
getReindexingTimelineView builds each interval config with a
ReindexingConfigBuilder whose baseTuningConfig is null, while
createCompactionJobs passes the template's tuningConfig into the same builder.
Any cascading supervisor with static tuning settings such as maxRowsInMemory,
splitHintSpec, maxNumConcurrentSubTasks, indexSpecForIntermediatePersists, or
retry/chat settings will have those settings in the actual compaction jobs but
not in the new reindexingTimeline API response or downloaded UI config. This
makes the preview/API incompatible with the real effective config. Pass
tuningConfig here as createCompactionJobs does, and add a test that a
non-partitionsSpec tuning field survives in the timeline view.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
