?????? [Kafka Streams] java.lang.IllegalArgumentException:Invalidtimestamp-1

2016-06-07 Thread ????
Hi Guozhang?? Thank you for your help. I got the reason. I am using logstash to push logs to kafka, and the logstash used an old producer in which messages didn't contain timestamp filed . I should improve logstash-kafka plugin to adapt to the kafka 0.10.0.0. Thank you. --

Re: Brokers changing mtime on data files during startup?

2016-06-07 Thread Ismael Juma
Also, it would be good to have a JIRA as it seems to be a 0.9 regression. Ismael On 6 Jun 2016 20:03, "Dustin Cote" wrote: > For those that have seen this issue on 0.9, can you provide some more > insight into your environments? What OS and filesystem are you running? > Do you find that you can

Problem with Kafka TLS configuration "no cipher suites in common"

2016-06-07 Thread Mick Mahoney
Trying to configure Topbeat client --> Kafka connection to use TLS. Both of these are on the same server but other clients will be on different servers. I have switched kafka debug on using the JVM '-Djavax.net.debug=ssl' switch and full debug is below. Amongst the debug is the ciper suites erro

Re: Does the Kafka Streams DSL support non-Kafka sources/sinks?

2016-06-07 Thread Gwen Shapira
Last time I checked (maybe 10 month ago), Camel was using the old async producer, which is not reliable (no callbacks!). Make sure they improved this before using it in a system where reliability is important. On Mon, Jun 6, 2016 at 9:44 PM, Asaf Mesika wrote: > I'd stay off the Camel. It's perfo

Re: Problem with Kafka TLS configuration "no cipher suites in common"

2016-06-07 Thread Ismael Juma
Hi Mick, This can happen if you are using a DSS key, but the client only supports RSA-based ciphers, see the following for more details: https://issues.apache.org/jira/secure/EditComment!default.jspa?id=12964371&commentId=15270520 Ismael On Tue, Jun 7, 2016 at 10:24 AM, Mick Mahoney wrote: >

Re: Problem with Kafka TLS configuration "no cipher suites in common"

2016-06-07 Thread Mick Mahoney
Hi Ismael, Thanks for the reply - i don't have permissions to see that link - is there any chance you can expand on it in the thread please ? Thanks, Mick On Tue, Jun 7, 2016 at 12:39 PM, Ismael Juma wrote: > Hi Mick, > > This can happen if you are using a DSS key, but the client only supports

Re: Problem with Kafka TLS configuration "no cipher suites in common"

2016-06-07 Thread Ismael Juma
Sorry, the link should have been (it's public, anyone can access): https://issues.apache.org/jira/browse/KAFKA-3647?focusedCommentId=15270520&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15270520 Ismael On Tue, Jun 7, 2016 at 12:43 PM, Mick Mahoney wrote: > Hi

Re: Problem with Kafka TLS configuration "no cipher suites in common"

2016-06-07 Thread Mick Mahoney
Perfect - thanks Ismael. Changed keytool -keystore kafka.server.keystore.jks -alias localhost -validity 365 -genkey to keytool -keystore kafka.server.keystore.jks -alias localhost -validity 365 -genkey -keyalg RSA work a treat :) Thanks again, Mick On Tue, Jun 7, 2016 at 12:51 PM, Is

RE: Problem with Kafka TLS configuration "no cipher suites in common"

2016-06-07 Thread Martin Gainty
MG>Ismael...any reason not to use openssl to generate the key and the cert simultaneously?openssl req -nodes -newkey rsa:2048 -keyout kafka.server.keystore.jks -out ca.certhttps://www.networking4all.com/en/support/ssl+certificates/manuals/apache/apache+http+server/generate+csr/ MG>Thanks MG>Mart

Re: HELP PLEASE->Kafka 0.9.0.0 create topic throwing ERROR kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0 with zookeeper 3.4.6

2016-06-07 Thread shubham
Hi , i am curious to know what was the issue, as i am facing the same problem. Regards, Shubham Agrawal

Serdes class

2016-06-07 Thread Saeed Ansari
Hi everyone, I have started to use KafkaStreams. I am using following dependency: org.apache.kafka kafka-streams 0.10.0.0 In the KafkaStream examples, there is *Serdes* class which does not include in this version. Is not this the latest version? Thank you, Saeed

Re: Serdes class

2016-06-07 Thread Eno Thereska
Hi Saeed, You'll also need to add a dependency on org.apache.kafka's kafka-clients, since that has the serialization classes. org.apache.kafka kafka-clients 0.10.0.0 Thanks Eno > On 7 Jun 2016, at 15:50, Saeed Ansari wrote: > > Hi everyone, > I h

Not able to monitor consumer group lag with new consumer and kerberos

2016-06-07 Thread Pierre LABIAUSSE
Hi, I'm not able to use kafka-consumer-groups.sh to monitor the lag of my consumers when my cluster is kerberized. I'm using kafka version 0.9.0 installed on an hortonworks hdp 2.4.0 cluster. I've replicated my setup on two sandboxes, one without kerberos and one with kerberos. On the one

Not able to monitor consumer group lag with new consumer and kerberos

2016-06-07 Thread Pierre Labiausse
Hi, I'm not able to use kafka-consumer-groups.sh to monitor the lag of my consumers when my cluster is kerberized. I'm using kafka version 0.9.0 installed on an hortonworks hdp 2.4.0 cluster. I've replicated my setup on two sandboxes, one without kerberos and one with kerberos. On the one witho

Invalid Version for API key

2016-06-07 Thread Chris Barlock
We are running some tests on upgrading from Kafka 2.10-0.8.2.1 to 2.11-0.10.00. So far, we only upgraded the server. All of the clients that I could verify are still using the Maven kafka-clients version 0.8.2.1. There are a number of exceptions in the server.log: [2016-06-07 16:00:00,266] E

Re: Not able to monitor consumer group lag with new consumer and kerberos

2016-06-07 Thread Harsha
Hi Pierre, Do you see any errors in the server.log when this command ran. Can you please open a thread here https://community.hortonworks.com/answers/index.html . Thanks, Harsha On Tue, Jun 7, 2016, at 09:22 AM, Pierre Labiausse wrote: > Hi, > > I'm not

KafkaStream and Kafka consumer group

2016-06-07 Thread Saeed Ansari
Hi, I have started a small Kafka streaming application. I need to assign partitions to consumers in a consumer group. I am not sure where to start and how to structure consumer groups in KafkaStreams. I found that there is a StreamPartitionAssignor that can be added to config, but still I do not k

Re: Brokers changing mtime on data files during startup?

2016-06-07 Thread Andrew Otto
Indeed! Just made one. https://issues.apache.org/jira/browse/KAFKA-3802 On Tue, Jun 7, 2016 at 4:26 AM, Ismael Juma wrote: > Also, it would be good to have a JIRA as it seems to be a 0.9 regression. > > Ismael > On 6 Jun 2016 20:03, "Dustin Cote" wrote: > > > For those that have seen this i

Broker keeps drifting into "Not leader for any partitions" state

2016-06-07 Thread Ashwin Jayaprakash
Hello, we have been running a 4 broker cluster for more than a year. Over the past few months one of our brokers tends to slip into a state where it is no longer a leader for any partition. The frequency at which this has started to happen is alarming. - We've been on version 2.10-0.8.2.1 for

Re: Not able to monitor consumer group lag with new consumer and kerberos

2016-06-07 Thread Ewen Cheslack-Postava
Pierre, I think you'll need the rest of the security-related configs you'd use for a normal client as well. You can use the --command-config flag to include additional settings stored in a property file. -Ewen On Tue, Jun 7, 2016 at 8:57 AM, Pierre LABIAUSSE wrote: > Hi, > > I'm not able to us