Re: Problem with timestamp in Producer

2016-10-30 Thread Debasish Ghosh
Thanks Matthias again for all the suggestions .. On Monday 31 October 2016, Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Just one comment to add: > > > it produces messages based > >> on 0.9.0 which obviously cannot be consumed by 0.10.0 brokers. > > Strictly sp

Re: Problem with timestamp in Producer

2016-10-30 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Just one comment to add: > it produces messages based >> on 0.9.0 which obviously cannot be consumed by 0.10.0 brokers. Strictly speaking, this is not correct. Brokers are compatible to older versions of clients (not the other way round though). Fo

Re: Problem with timestamp in Producer

2016-10-30 Thread Debasish Ghosh
I think I found out what happened .. I was installing Kafka under DC/OS on AWS and following this doc https://dcos.io/docs/1.8/usage/tutorials/kafka/ for kafka installation. This works fine and installs kafka version 0.10.0. But in the example where it shows how to produce and consume messages, th

Re: Problem with timestamp in Producer

2016-10-30 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 The simplest way should be to check the java classpath. Insert an echo $CLASSPATH at the end of bin/kafka-run-class.sh Than run bin/kafka-console-producer.sh with no argument. You should see the classpath be printed out. Look for 'kafka-clients-

Re: Problem with timestamp in Producer

2016-10-29 Thread Debasish Ghosh
Hello Mathias - Regarding .. In case you do have 0.10 brokers, it might however happen, that bin/kafka-console-producer.sh > does use 0.9 producer. How can I check this ? Thanks! On Sat, Oct 29, 2016 at 12:23 PM, Debasish Ghosh wrote: > I agree .. the problem is DC/OS still ships the older

Re: Problem with timestamp in Producer

2016-10-28 Thread Debasish Ghosh
I agree .. the problem is DC/OS still ships the older version. Let me check if I can upgrade this .. Thanks! On Sat, Oct 29, 2016 at 12:21 PM, Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Btw: I would highly recommend to use Kafka 0.10.1 -- there are many > new

Re: Problem with timestamp in Producer

2016-10-28 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Btw: I would highly recommend to use Kafka 0.10.1 -- there are many new Streams feature and usability improvements and bug fixes. - -Matthias On 10/28/16 11:42 PM, Matthias J. Sax wrote: > That sounds reasonable. However, I am wondering how your St

Re: Problem with timestamp in Producer

2016-10-28 Thread Debasish Ghosh
I will check out all options that u mentioned. I am sure on my local it's all 0.10.0, so no wonder it works correctly. In The cluster, I just checked the version of Kafka that ships with DC/OS 1.8 (the version I get with dcos package install kafka) is 0.9.0. Regarding .. In case you do have 0.10 b

Re: Problem with timestamp in Producer

2016-10-28 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 That sounds reasonable. However, I am wondering how your Streams application can connect to 0.9 broker in the first place. Streams internally uses standard Kafka clients, and those are not backward compatible. Thus, the 0.10 Streams clients should no

Re: Problem with timestamp in Producer

2016-10-28 Thread Debasish Ghosh
Hello Mathias - Thanks a lot for the response. I think what may be happening is a version mismatch between the development & deployment versions of Kafka. The Kafka streams application that I developed uses 0.10.0 based libraries. And my local environment contains a server installation of the same

Re: Problem with timestamp in Producer

2016-10-28 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hey, we just added a new FAQ entry for upcoming CP 3.2 release that answers your question. I just c&p it here. More concrete answer below. > If you get an exception similar to the one shown below, there are > multiple possible causes: > > Exceptio

Re: Problem with timestamp in Producer

2016-10-28 Thread Debasish Ghosh
I am actually using 0.10.0 and NOT 0.10.1 as I mentioned in the last mail. And I am using Kafka within a DC/OS cluster under AWS. The version that I mentioned works ok is on my local machine using a local Kafka installation. And it works for both single broker and multi broker scenario. Thanks.