aho135 commented on code in PR #19571: URL: https://github.com/apache/druid/pull/19571#discussion_r3399352719
########## docs/ingestion/kafka-ingestion.md: ########## @@ -263,6 +263,50 @@ The following example shows a supervisor spec with idle configuration enabled: ``` </details> +#### Streaming partitions spec + +When you set `streamingPartitionsSpec.partitionDimensions` in the tuning config, the supervisor tracks the distinct values observed for each listed dimension during ingestion. At segment publish time, each segment is annotated with only the values it actually ingested. The broker then uses these annotations to skip segments at query time when the query filter doesn't intersect the segment's declared values. Review Comment: Are you thinking to add additional properties into streamingPartitionsSpec in the future? Wondering if just `tuningConfig.partitionDimensions` would be more clear, and we can just add to partitionDimensions as needed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
