Thanks Tom. Sounds good. :) Ismael
On Thu, Jun 22, 2017 at 4:21 PM, Tom Crayford <tcrayf...@heroku.com> wrote: > Hi Ismal, > > Sure. It's a standard heroku plan, the `extended-2`, which has 8 brokers. > We did several rounds of performance testing, some with low numbers of > partitions and topics (16 partitions with one topic, which we typically see > the highest throughput on) and some with many more (many thousands of > partitions and topics). Replication factor was always 3. Partitions per > broker was roughly equal in each round (we just use the inbuilt kafka > scripts to create topics). Lastly, all tests run were with SSL enabled for > both client and inter-broker traffic. > > These are the same performance tests we've run for previous versions, and > wrote about when testing 0.10 here: https://blog.heroku.com/ > apache-kafka-010-evaluating-performance-in-distributed-systems > > In all performance tests we've ever done, Kafka has been network limited, > not disk or CPU. This, I think is common for Kafka setups and workloads, > but it does change significantly with what hardware you're using. > > Because our performance tests run across cloud instances where there might > be contention and variability, we typically do a multiple runs on different > clusters with each setup before reporting results. > > Thanks > > Tom Crayford > Heroku Kafka > > On Thu, Jun 22, 2017 at 4:08 PM, Ismael Juma <ism...@juma.me.uk> wrote: > >> Hi Tom, >> >> Would you be able to share the details of the Kafka Cluster that you used >> for performance testing? We are interested in the number of brokers, >> topics, partitions per broker and replication factor. Thanks! >> >> Ismael >> >> On Mon, Jun 19, 2017 at 3:15 PM, Tom Crayford <tcrayf...@heroku.com> >> wrote: >> >>> Hello, >>> >>> Heroku has been testing 0.11.0.0 RC0, mostly focussed on backwards >>> compatibility and performance. So far, we note a slight performance >>> increase from older versions when using not-current clients >>> >>> Testing a 0.9 client against 0.10.2.1 vs 0.11.0.0 rc0: 0.11.0.0 rc0 has >>> slightly higher throughput for both consumers and producers. We expect >>> this >>> because the message format improvements should lead to greater >>> efficiency. >>> >>> We have not yet tested a 0.11.0.0 rc0 client against a 0.11.0.0 rc0 >>> cluster, because our test setup needs updating for that. >>> >>> We've tested simple demo apps against 0.11.0.0rc0 (that we've run against >>> older clusters): >>> http://github.com/heroku/heroku-kafka-demo-ruby >>> https://github.com/heroku/heroku-kafka-demo-node >>> https://github.com/heroku/heroku-kafka-demo-java >>> https://github.com/heroku/heroku-kafka-demo-go >>> >>> This comprises a range of community supported clients: ruby-kafka, >>> no-kafka, the main JVM client and sarama. >>> >>> We didn't see any notable issues there, but it's worth noting that all of >>> these demo apps do little more than produce and consume messages. >>> >>> We have also tested failure handling in 0.11.0.0 rc0, by terminating >>> nodes. >>> Note that this does *not* test any of the new exactly-once features, just >>> "can I terminate a broker whilst producing to/consuming from the cluster. >>> We see the same behaviour as 0.10.2.1 there, just a round of errors from >>> the client, like this: >>> >>> org.apache.kafka.common.errors.NetworkException: The server disconnected >>> before a response was received. >>> >>> but that's expected. >>> >>> We have tested creating and deleting topics heavily, including deleting a >>> topic in the middle of broker failure (the controller waits for the >>> broker >>> to come back before being deleted, as expected) >>> >>> We have also tested upgrading a 0.10.2.1 cluster to 0.11.0.0 rc0 without >>> issue >>> >>> We have also tested partition preferred leader election (manual, with the >>> admin script), and partition rebalancing to grow and shrink clusters. >>> >>> We have not yet tested the exactly once features, because various core >>> committers said that they didn't expect this feature to be perfect in >>> this >>> release. We expect to test this this week though. >>> >>> Given that the blockers fixed between RC0 and RC1 haven't changed much in >>> the areas we tested, I think the positive results here still apply. >>> >>> Thanks >>> >>> Tom Crayford >>> Heroku Kafka >>> >>> On Thu, Jun 8, 2017 at 2:55 PM, Ismael Juma <ism...@juma.me.uk> wrote: >>> >>> > Hello Kafka users, developers and client-developers, >>> > >>> > This is the first candidate for release of Apache Kafka 0.11.0.0. It's >>> > worth noting that there are a small number of unresolved issues >>> (including >>> > documentation and system tests) related to the new AdminClient and >>> > Exactly-once functionality[1] that we hope to resolve in the next few >>> days. >>> > To encourage early testing, we are releasing the first release >>> candidate >>> > now, but there will be at least one more release candidate. >>> > >>> > Any and all testing is welcome, but the following areas are worth >>> > highlighting: >>> > >>> > 1. Client developers should verify that their clients can >>> produce/consume >>> > to/from 0.11.0 brokers (ideally with compressed and uncompressed data). >>> > Even though we have compatibility tests for older Java clients and we >>> have >>> > verified that librdkafka works fine, the only way to be sure is to test >>> > every client. >>> > 2. Performance and stress testing. Heroku and LinkedIn have helped with >>> > this in the past (and issues have been found and fixed). >>> > 3. End users can verify that their apps work correctly with the new >>> > release. >>> > >>> > This is a major version release of Apache Kafka. It includes 32 new >>> KIPs. >>> > See >>> > the release notes and release plan (https://cwiki.apache.org/ >>> > confluence/display/KAFKA/Release+Plan+0.11.0.0) for more details. A >>> few >>> > feature highlights: >>> > >>> > * Exactly-once delivery and transactional messaging >>> > * Streams exactly-once semantics >>> > * Admin client with support for topic, ACLs and config management >>> > * Record headers >>> > * Request rate quotas >>> > * Improved resiliency: replication protocol improvement and >>> single-threaded >>> > controller >>> > * Richer and more efficient message format >>> > >>> > Release notes for the 0.11.0.0 release: >>> > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc0/RELEASE_NOTES.html >>> > >>> > Kafka's KEYS file containing PGP keys we use to sign the release: >>> > http://kafka.apache.org/KEYS >>> > >>> > * Release artifacts to be voted upon (source and binary): >>> > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc0/ >>> > >>> > * Maven artifacts to be voted upon: >>> > https://repository.apache.org/content/groups/staging/org/apache/kafka/ >>> > >>> > * Javadoc: >>> > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc0/javadoc/ >>> > >>> > * Tag to be voted upon (off 0.11.0 branch) is the 0.11.0.0 tag: >>> > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h= >>> > d5ee02b187fafe08b63deb52e6b07c8d1d12f18d >>> > >>> > * Documentation: >>> > http://kafka.apache.org/0110/documentation.html >>> > >>> > * Protocol: >>> > http://kafka.apache.org/0110/protocol.html >>> > >>> > * Successful Jenkins builds for the 0.11.0 branch: >>> > Unit/integration tests: https://builds.apache.org/job/ >>> > kafka-0.11.0-jdk7/121/ >>> > >>> > Thanks, >>> > Ismael >>> > >>> > [1] https://issues.apache.org/jira/issues/?jql=project%20% >>> > 3D%20KAFKA%20AND%20fixVersion%20%3D%200.11.0.0%20AND%20resol >>> ution%20%3D% >>> > 20Unresolved%20ORDER%20BY%20due%20ASC%2C%20priority% >>> > 20DESC%2C%20created%20ASC >>> > >>> >> >> >
