Hi Matthias, Yes, I'm aware of Wall-clock time punctuation, but that is not the behavior my customers want, so I can't use that unfortunately...
OK. I'm going to try and write a utility "StreamTimeInspector" transformer that will do as you suggest and give me the stats the I want. If we find this useful at my company I'll consider submitting it as a PR. Is there any current precedent around transformers like this in KafkaStreams? Transformers you insert for interrogation only? Dan On Mon, Apr 2, 2018 at 2:10 PM Matthias J. Sax <matth...@confluent.io> wrote: > Dan, > > Kafka Streams supports two types of punctuation: event-time and > wall-clock time punctuation. Wall-clock time punctuation will be called > even if event-time does not progress and even if there is no new input > data available for processing. Not sure if this is what you are looking > for. > > Beside this, Processor API exposes record metadata information. Thus, > you can plugin a custom `transform()` step to access and maintain the > information/stats you are interested in (The provided `context` object > from `init()` is your friend for this.) and maybe make it queryable via > Interactive Queries? > > Hope this helps. > > > -Matthias > > On 4/2/18 11:26 AM, dan bress wrote: > > I have an app that is consuming multiple topics, and punctuating on > stream > > time. I know that punctuate is driven by the min time of all the > > partitions of all the topics driving the transformer that I am > punctuating > > on. When I deploy my app and punctuate is not called as I expect, what > > tools do I have to understand where time is per > > instance/thread/topic/partition? Does Kafka Streams expose stats for > this? > > > > I would like something like a stat of something like: > > > > Kafka Streams Instance / Kafka Stream Thread / Topic / Partition / > > EventTime = 1522693495291 > > > > Then I can look at which topic/partition is lagging to debug my problem. > > > > Does something like this exist? Is this a reasonable feature request? > > > > Thanks! > > Dan > > > >
