Hi, I’m sorry for the trouble. It looks like it was a mistake during
https://github.com/apache/kafka/pull/6521 Specifically, while addressing code review comments to change a bunch of other logs from debugs to warnings, that one seems to have been included by accident: https://github.com/apache/kafka/commit/ac27e8578f69d60a56ba28232d7e96c76957f66c I’ll see if I can fix it today. Regarding Bruno's thoughts, there was a pretty old decision to capture the "skipped records" as a metric for visibility and log it at the debug level for debuggability. We decided that "warning" wasn't the right level because Streams is operating completely as specified. However, I do agree that it doesn't seem right to see more skipped records during start-up; I would expect to see exactly the same records skipped during start-up as during regular processing, since the skipping logic is completely deterministic and based on the sequence of timestamps your records have in the topic. Maybe you just notice it more during startup? I.e., if there are 1000 warning logs spread over a few months, then you don't notice it, but when you see them all together at start-up, it's more concerning? Thanks, -John On Mon, Feb 10, 2020, at 10:15, Bruno Cadonna wrote: > Hi, > > I am pretty sure this was intentional. All skipped records log > messages are on WARN level. > > If a lot of your records are skipped on app restart with this log > message on WARN-level, they were also skipped with the log message on > DEBUG-level. You simply did not know about it before. With an > in-memory window store, this message is logged when a window with a > start time older than the current stream time minus the retention > period is put into the window store, i.e., the window is NOT inserted > into the window stroe. If you get a lot of them on app restart, you > should have a look at the timestamps of your records and the retention > of your window store. If those values do not explain the behavior, > please try to find a minimal example that shows the issue and post it > here on the mailing list. > > On Mon, Feb 10, 2020 at 2:27 PM Samek, Jiří <sa...@avast.com.invalid> wrote: > > > > Hi, > > > > in > > https://github.com/apache/kafka/commit/9f5a69a4c2d6ac812ab6134e64839602a0840b87#diff-a5cfe68a5931441eff5f00261653dd10R134 > > > > log level of "Skipping record for expired segment" was changed from debug > > to warn. Was it intentional change? Should it be somehow handled by user? > > How can user handle it? I am getting a lot of these on app restart. >
