Re: Build failed in Jenkins: kafka-trunk-jdk11 #1369
ask :streams:upgrade-system-tests-21:compileTestJava > > Task :streams:upgrade-system-tests-21:processTestResources NO-SOURCE > > Task :streams:upgrade-system-tests-21:testClasses > > Task :streams:upgrade-system-tests-21:checkstyleTest > > Task :streams:upgrade-system-tests-21:spotbugsMain NO-SOURCE > > Task :streams:upgrade-system-tests-21:test > > Task :streams:upgrade-system-tests-22:compileJava NO-SOURCE > > Task :streams:upgrade-system-tests-22:processResources NO-SOURCE > > Task :streams:upgrade-system-tests-22:classes UP-TO-DATE > > Task :streams:upgrade-system-tests-22:checkstyleMain NO-SOURCE > > Task :streams:upgrade-system-tests-22:compileTestJava > > Task :streams:upgrade-system-tests-22:processTestResources NO-SOURCE > > Task :streams:upgrade-system-tests-22:testClasses > > Task :streams:upgrade-system-tests-22:checkstyleTest > > Task :streams:upgrade-system-tests-22:spotbugsMain NO-SOURCE > > Task :streams:upgrade-system-tests-22:test > > Task :streams:upgrade-system-tests-23:compileJava NO-SOURCE > > Task :streams:upgrade-system-tests-23:processResources NO-SOURCE > > Task :streams:upgrade-system-tests-23:classes UP-TO-DATE > > Task :streams:upgrade-system-tests-23:checkstyleMain NO-SOURCE > > Task :streams:upgrade-system-tests-23:compileTestJava > > Task :streams:upgrade-system-tests-23:processTestResources NO-SOURCE > > Task :streams:upgrade-system-tests-23:testClasses > > Task :streams:upgrade-system-tests-23:checkstyleTest > > Task :streams:upgrade-system-tests-23:spotbugsMain NO-SOURCE > > Task :streams:upgrade-system-tests-23:test > > Task :streams:upgrade-system-tests-24:compileJava NO-SOURCE > > Task :streams:upgrade-system-tests-24:processResources NO-SOURCE > > Task :streams:upgrade-system-tests-24:classes UP-TO-DATE > > Task :streams:upgrade-system-tests-24:checkstyleMain NO-SOURCE > > Task :streams:upgrade-system-tests-24:compileTestJava > > Task :streams:upgrade-system-tests-24:processTestResources NO-SOURCE > > Task :streams:upgrade-system-tests-24:testClasses > > Task :streams:upgrade-system-tests-24:checkstyleTest > > Task :streams:upgrade-system-tests-24:spotbugsMain NO-SOURCE > > Task :streams:upgrade-system-tests-24:test > > Task :streams:upgrade-system-tests-25:compileJava NO-SOURCE > > Task :streams:upgrade-system-tests-25:processResources NO-SOURCE > > Task :streams:upgrade-system-tests-25:classes UP-TO-DATE > > Task :streams:upgrade-system-tests-25:checkstyleMain NO-SOURCE > > Task :streams:upgrade-system-tests-25:compileTestJava > > Task :streams:upgrade-system-tests-25:processTestResources NO-SOURCE > > Task :streams:upgrade-system-tests-25:testClasses > > Task :streams:upgrade-system-tests-25:checkstyleTest > > Task :streams:upgrade-system-tests-25:spotbugsMain NO-SOURCE > > Task :streams:upgrade-system-tests-25:test > > FAILURE: Build failed with an exception. > > * What went wrong: > Execution failed for task ':core:test'. > > There were failing tests. See the report at: file://< > https://builds.apache.org/job/kafka-trunk-jdk11/ws/core/build/reports/tests/test/index.html > > > > * Try: > Run with --stacktrace option to get the stack trace. Run with --info or > --debug option to get more log output. Run with --scan to get full insights. > > * Get more help at https://help.gradle.org > > Deprecated Gradle features were used in this build, making it incompatible > with Gradle 7.0. > Use '--warning-mode all' to show the individual deprecation warnings. > See > https://docs.gradle.org/6.3/userguide/command_line_interface.html#sec:command_line_warnings > > BUILD FAILED in 1h 45m 58s > 215 actionable tasks: 183 executed, 32 up-to-date > Build step 'Execute shell' marked build as failure > Recording test results > ERROR: No tool found matching GRADLE_4_10_2_HOME > Setting GRADLE_4_10_3_HOME=/home/jenkins/tools/gradle/4.10.3 > ERROR: No tool found matching GRADLE_4_10_2_HOME > Setting GRADLE_4_10_3_HOME=/home/jenkins/tools/gradle/4.10.3 > Not sending mail to unregistered user nore...@github.com > -- Kind Regards, Shafaq
Kafka AWS deployment + UI console
We are going to deploy Kafka in Production and also monitor it via console. (e.g. State of partitions in Broker- leader and slaves, state of consumers ) Is there out-of-the-box solution? What is the best and efficient way of deployment and monitoring Has someone tried this- looks promising http://www.michael-noll.com/blog/2014/03/17/wirbelsturm-one-click-deploy-storm-kafka-clusters-with-vagrant-puppet/ -- Kind Regards, Shafaq
Fwd: Questions about failure recovery, load balancing and producer ack/async mode
Hi, Following need more elaboration after reading kafka docs: 1- In a scenario during leader fails over, what happens to messages that are not committed to other followers and to the messages that producer keep in sending (in async mode) till new leader is elected. Can the producer buffer these messages till the failover gets completed (Do we see exceptions. Are these all lost? 2. Broker load distribution or to be exact partition load distribution is done via hashed key mod no of partition, is that correct? How is it done at the consumer side where one thread per partition (for high-level consumer) How is the faliover handlled 3. How do we configure producer for wait for leader ack and wait for commited act which Jun Rao talks about in his 2013 apachecon presentation -- Kind Regards, Shafaq -- Kind Regards, Shafaq
0.8 Head: Simple consumer no receiving messages
Hi,
I see the following scenario:
1. Send messages under some topic X, able to see the log folder in Kafka
Broker with name X-0 (Zeroth partition) and having files xxx.log and
xxx.index under them. So guess this is fine
2. THen I fire up the consumer for topic X, it is able to find two streams
(mapping to two partitions I have defined).
Map>> consumerMap =
consumer
.createMessageStreams(topicCountMap);
List> streams = null;
Iterator it = topicCountMap.keySet().iterator();
int threadNumber= 0;
while(it.hasNext()) {
String topic = it.next();
streams = consumerMap.get(topic);
for (KafkaStream stream : streams) {
System.out.println("threadNo =" + threadNumber + " for
topic = " + topic );
new ConsumerThreadRunnable(stream, threadNumber, topic));
threadNumber++;
}
}
However I don't get any messages in the CounsmerTHreanRunnable here
ConsumerIterator it = stream.iterator();
while (it.hasNext() ) {
byte[] nextMessageByteArray = it.next().message();
}
If I start the consumer first and then restart the producer thread, sending
the messages for topic X then consumer is able to receive the messages.
>From kafka docs the high-level consumer thread does long polling till the
message is available.
What is wrong I'm doing? Any idea to get around the problem.
thanks!
--
Kind Regards,
Shafaq
Leader no available for partiton error
(KafkaApis.scala:452) at kafka.server.KafkaApis.handle(KafkaApis.scala:69) at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42) at java.lang.Thread.run(Thread.java:724) -- Kind Regards, Shafaq
