vamossagar12 commented on pull request #10798: URL: https://github.com/apache/kafka/pull/10798#issuecomment-872398600
@guozhangwang , thats the commit i was referring to. Setting a byte order and seems to have an impact on performance. Also, earlier i was calling put first and then calling flip which is wrong. So, I tweaked that. I have another update on this piece. I created a small kafka streams app which subscribes to a single partition topic and for every record consumed, it has 2 processors: 1) putAll processor which ingests 1M records into state store and then 2). range processor which does a range query for all keys inserted. I pushed 100 records to the topic and these 2 processors got invoked each time. Here are the numbers from that run: ``` DirectByteBuffer -> Operator: putAll, Real: 80.722 CPU: 273.690 GC: 6.974 GCCount: 36 avg throughput: 1269130.717 op/s p95 throughput: 1430952.065 op/s p99 throughput: 1453862.308 op/s Operator: range, Real: 114.511 CPU: 271.410 GC: 5.501 GCCount: 26 avg throughput: 920462.473 op/s p95 throughput: 1234215.537 op/s p99 throughput: 1588364.216 op/s ``` ``` Original -> Operator: putAll, Real: 92.988 CPU: 288.550 GC: 6.900 GCCount: 53 avg throughput: 1088404.000 op/s p95 throughput: 1233248.141 op/s p99 throughput: 1252514.840 op/s Operator: range, Real: 110.418 CPU: 268.070 GC: 6.105 GCCount: 40 avg throughput: 957558.135 op/s p95 throughput: 1277487.581 op/s p99 throughput: 1388059.720 op/s ``` The way I calculate throughput is for every iteration, I divided 1M by time taken for that iteration. You can find the relevant code here: https://github.com/vamossagar12/kafka-streams-throughputbenchmarking/tree/master/src/main/java/com/bytebyffer/benchmarks Let me know if you and @cadonna think we need other. kinds of benchmarks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
