Re: [DISCUSS] KIP-363

Hi, I'd like to pick up this conversation again, now that all the RPCs are using the generated classes. Does anyone have any more questions or comments about this? Cheers, Tom On Mon, Jul 13, 2020 at 2:48 PM Tom Bentley wrote: > Hi Magnus, > > Sorry for the late reply. In the specific case y

Re: [DISCUSS] KIP-363

Hi Magnus, Sorry for the late reply. In the specific case you mentioned of an OffsetCommitResponse with 100 partitions across 10 topics we end up with -200bytes (no error) and +200bytes (any error) in absolute terms. Precise numbers depend on topic names too, of course but I did a little test usin

Re: [DISCUSS] KIP-363

Hi Tom, I think it would be useful with some real world (or made up!) numbers on how much relative/% space is saved for the most error-dense protocol requests. E.g., an OffsetCommitResponse with 10 topics and 100 failing partitions would reduce the overall size by % bytes. Thanks, Magnus Den ti

Re: [DISCUSS] KIP-363

Hi Tom, Thanks for this. I think the tough part is probably the few messages that are still using manual serialization, which can't be easily converted to using this. So we will probably have to upgrade them to using automatic generation, or accept a little inconsistency for a while until the

[DISCUSS] KIP-363

Hi all, Following a suggestion from Colin in the KIP-625 discussion thread, I'd like to start discussion on a much smaller KIP which proposes to make error codes and messages tagged fields in all RPCs. https://cwiki.apache.org/confluence/display/KAFKA/KIP-363%3A+Make+RPC+error+codes+and+messages+

Re: [DISCUSS] KIP-363: Allow performance tools to print final results to output file

Hi all, I have updated the KIP based on the suggestions received so far, please take a look at the new version: https://cwiki.apache.org/confluence/display/KAFKA/KIP-363%3A+Allow+performance+tools+to+print+final+results+to+output+file Best, - Attila On Fri, Aug 24, 2018 at 3:52 PM Attila Sasvári

Re: [DISCUSS] KIP-363: Make FunctionConversions private

I'm also in favor of this. I don't think it's controversial either. Should we just move to a vote? On Thu, Aug 23, 2018 at 7:01 PM Guozhang Wang wrote: > +1. > > On Thu, Aug 23, 2018 at 12:47 PM, Ted Yu wrote: > > > +1 > > > > In the Motivation section, you can quote the comment from pull reque

Re: [DISCUSS] KIP-363: Allow performance tools to print final results to output file

Thanks for your feedback, Kevin & Viktor! I will update the KIP next week. @Kevin - I also had to write sed based one-liners when I ran performance tests to extract results (that were displayed in some charts later). 1. Initially, I wanted to overwrite any exisisting file and document this behavio

Re: [DISCUSS] KIP-363: Allow performance tools to print final results to output file

Hi Attila, Thanks for the KIP, I think overall it looks good. I have three comments: 1. Would you mind adding an example? (Later on we'd need anyway for the public doc.) 2. Do you want to add any 3rd party CSV reader/writer library or will you implement that too? 3. What is the separator or is tha

Re: [DISCUSS] KIP-363: Allow performance tools to print final results to output file

Hi Attila, Thanks for the KIP. I think this would be a useful feature. Every time I have to benchmark using these performance tools, I end up redirecting the output to a file anyways. Just a couple minor questions... 1. If the configured file already exists, what would be the outcome? My intuit

Re: [DISCUSS] KIP-363: Make FunctionConversions private

+1. On Thu, Aug 23, 2018 at 12:47 PM, Ted Yu wrote: > +1 > > In the Motivation section, you can quote the comment from pull request so > that reader doesn't have to click through. > > Cheers > > On Thu, Aug 23, 2018 at 12:13 PM Joan Goyeau wrote: > > > Hi, > > > > As pointed out in this comment

Re: [DISCUSS] KIP-363: Make FunctionConversions private

+1 In the Motivation section, you can quote the comment from pull request so that reader doesn't have to click through. Cheers On Thu, Aug 23, 2018 at 12:13 PM Joan Goyeau wrote: > Hi, > > As pointed out in this comment #5539 (comment) >

[DISCUSS] KIP-363: Make FunctionConversions private

Hi, As pointed out in this comment #5539 (comment) the object FunctionConversions is only of internal use and therefore should be private to the lib only so that we can do changes without going through KIP like this one. KIP: https

[DISCUSS] KIP-363: Allow performance tools to print final results to output file

Hi all, I have created a minor KIP to allow consumer and producer performance tools to print final results to output file in CSV format. https://cwiki.apache.org/confluence/display/KAFKA/KIP-363%3A+Allow+performance+tools+to+print+final+results+to+output+file Please take a look and share your th