Jon, the windowing operation of Kafka's Streams API (in its DSL) aligns time-based windows to the epoch [1]:
Quoting from e.g. hopping windows (sometimes called sliding windows in other technologies): > Hopping time windows are aligned to the epoch, with the lower interval bound > being inclusive and the upper bound being exclusive. “Aligned to the epoch” > means that the first window starts at timestamp zero. > For example, hopping windows with a size of 5000ms and an advance interval > (“hop”) of 3000ms have predictable window boundaries `[0;5000),[3000;8000),...` > — and not `[1000;6000),[4000;9000),...` or even something “random” like > `[1452;6452),[4452;9452),...`. Would that help you? -Michael [1] http://docs.confluent.io/current/streams/developer-guide.html On Mon, Mar 20, 2017 at 12:51 PM, Jon Yeargers <jon.yearg...@cedexis.com> wrote: > Is this possible? Im wondering about gathering data from a stream into a > series of windowed aggregators: minute, hour and day. A separate process > would start at fixed intervals, query the appropriate state store for > available values and then hopefully clear / zero / reset everything for the > next interval. > > I could use the retention period setting but I would (somehow) need to > guarantee that the windows would reset on clock boundaries and not based on > start time for the app. >
