Hi Peter, Thanks for the reply! I noticed that, after deployment, changelog topic has high bytes in/sec and messages/sec, but low bytes out/sec. Once the app is unstable, or traffic is bumpy, it switched: changelog topic has low bytes in/sec and messages/sec but high bytes out/sec. Is it normally? Why it will switch during unstable time?
Thanks! On Thu, Oct 3, 2019 at 5:03 AM Peter Levart <peter.lev...@gmail.com> wrote: > > Hi Hu, > > On 10/2/19 8:54 PM, Xiyuan Hu wrote: > > Hi All, > > > > I'm doing smoke testing with my Kafka Streams app(V2.1.0). I noticed > > that below behaviors: > > 1) Out throughput of changelog topic could go up to 70mb/s while the > > in-traffic is around 10mb/s. > > 2) When traffic is bumpy, either due to producer/consumer throttle or > > some other reasons I'm still debugging, changelog topic out throughput > > could go up to 160mb/s while in-throughput is less than 1mb/s. And > > from the monitor tool, no messages are actually put into this topic. > > > > Could anyone help me to understand how changelog topic works? Given it > > still has consumer lag and producer timeout issues(records > > expired(5min) since batch created), is it just the cluster > > bandwidth/capacity issue? > > > > Thanks, appreciate all the help!! > > In case you're using KeyValueStore (the simplest of stores), each update > of a (K, V) entry into the store produced by the processor of your > stream will result in one message written to the log topic of that > store. Multiple quick consecutive updates to the same entry (with same > key) are possibly collapsed into just one message holding the last > version of the entry. So it is possible that your input stream is having > keys that are not repeating very frequently in short time, therefore the > rate of messages/s is roughly the same in your input and log topics. The > difference in bytes/s therefore must be the result of input topic > message size vs. log topic message size. The log topic message is > basically the whole (K, V) entry in the KVStore. > > Regards, Peter >
