Hi Chia-Ping Earlier in the discussion phase, Federico Valeri has also proposed the idea of a "producer autopilot" which could switch to a steady state mode once the p99 was acceptably stable. I agree with both of you that an automatic warmup would be a very useful addition to the producer performance tooling, but creating a universal definition of "stable" and the automatic detection of stability are both somewhat complex. For these reasons, I think it's best that automatic warmup is contained in a follow-on KIP. I've attempted to include your ideas in the Rejected Alternatives section of the KIP.
Thanks, Matt -----Original Message----- From: Chia-Ping Tsai <chia7...@apache.org> Sent: Tuesday, September 24, 2024 4:22 AM To: dev@kafka.apache.org Subject: RE: [VOTE] KIP-1052: Enable warmup in producer performance test hi Matt Apologies for the delayed response. I completely agree that ProducerPerformance should separate warmup statistics from steady-state metrics. Overall +1, but I have a small question: Have you considered implementing an explicit check for warmup instead of relying on sending a set number of records? My concern is that users may not know how many records are necessary to complete the warmup. Rather than relying on a rule of thumb, ProducerPerformance could check the metadata and node latency (via metrics) to ensure that the node information (such as connection, DNS, and metadata) is ready. In summary, this approach introduces a flag called enable-warmup instead of using warmup-records. The advantage is that users no longer need to specify the number of warmup records. When the flag is enabled, ProducerPerformance will continue sending warmup records until the node information is fully ready. Best, Chia-Ping On 2024/09/04 22:20:15 "Welch, Matt" wrote: > Hi Kafka devs, > > Bumping this VOTE thread again for visibility. > > Thanks, > Matt > > -----Original Message----- > From: Welch, Matt <matt.we...@intel.com> > Sent: Friday, August 23, 2024 4:26 PM > To: dev@kafka.apache.org > Subject: RE: [VOTE] KIP-1052: Enable warmup in producer performance > test > > Hi Kafka devs, > > Bumping this VOTE thread for visibility. > > Thanks, > Matt > > -----Original Message----- > From: Federico Valeri <fedeval...@gmail.com> > Sent: Monday, August 19, 2024 12:38 AM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-1052: Enable warmup in producer performance > test > > Hi Matt, +1 (non binding) from me. Thanks! > > Just a suggestion: I think that the following output line does not add much > value and could be removed. > > "Warmup first 100000 records. Steady-state results will print after the > complete-test summary." > > On Wed, Aug 14, 2024 at 8:06 PM Welch, Matt <matt.we...@intel.com> wrote: > > > > > > Hi all, > > > > It seems discussion has been quiet for a couple of weeks so I'd like > > to call a vote on KIP-1052 > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1052%3A+Enable > > +w > > armup+in+producer+performance+test > > > > Thanks, > > Matt Welch > > >
