Hi,
I am using a kafka spout to read data into Storm 0.9.2.
whenever i run my program and check the console consumer, i can see that
there is an aerror saying
*" Failed to add leader for partitions [Json1234,0]; will retry
(kafka.consumer.ConsumerFetcherManager$LeaderFinderThread"*
along with a lo
Hi Francois,
Sorry in getting back to you a bit late.
I am still not able to trace the problem.
No i don't have a partition key and its not respecting "
topic.metadata.refresh.interval.ms" , even if i set this to 0, data is
only being produced to only one partition.
Ashutosh
On Mon, Aug 25, 2
Hi,
While I am not sure that JDK8 is the problem, what I did is simply clone
and gardle the source.
I kept getting failures and excluding tasks until eventually I did this:
*gradle -PscalaVersion=2.9.2 -x :clients:javadoc -x :clients:signArchives
-x :clients:licenseTest -x :contrib:signArchives cl
Hi ,
i have a topology where i am emmitting json objects onto a kafka topic
using a kafka producer to a storm bolt.
I have been relaying the same objects on the same topic over and over again
as i am figuring out few things.
after a while i can see that if i use kafkaConfig.forceFromStart=true; th
what gradle version is used to build kafka_2.9.2-0.8.1.1 ?
tried with v2 and failed with :
gradle --stacktrace clean
FAILURE: Build failed with an exception.
* Where:
Build file
'/home/shlomi/0dec0xb/project/vpmb/master/3rdparty/kafka/code/kafka-0.8.1.1-src/build.gradle'
line: 34
* What went
Hi Siddharth,
Please check if your kafka producer is still running.
Regards,
Dhaval Modi
dhavalmod...@gmail.com
On 4 September 2014 19:39, siddharth ubale
wrote:
> Hi ,
>
> i have a topology where i am emmitting json objects onto a kafka topic
> using a kafka producer to a storm bolt.
> I h
Did you tried "gradlew" script in kafka source dir.
-Harsha
On Thu, Sep 4, 2014, at 07:32 AM, Shlomi Hazan wrote:
> what gradle version is used to build kafka_2.9.2-0.8.1.1 ?
>
> tried with v2 and failed with :
>
>
>
> gradle --stacktrace clean
>
> FAILURE: Build failed with an exception.
>
it failed with JDK 8 so I hoped a newer gradle will maybe do the magic, and
stepped into this other problem.
I assume you will say :
install JDK 7 and build with our gradle 1.6.
is it so?
Shlomi
On Thu, Sep 4, 2014 at 5:41 PM, Harsha wrote:
> Did you tried "gradlew" script in kafka source dir.
0.7.2 is quite old. The most recent 0.8.1.1 release has better ide support
through gradle.
Thanks,
Jun
On Tue, Sep 2, 2014 at 8:53 AM, Hemanth Yamijala wrote:
> To give some more details:
>
> * I used the src tarball of Kafka 0.7.2.
> * Ran sbt idea. This generated a bunch of *.iml files unde
When building you need to use the ./gradelw script as Harsha said. Please
take a look at the README for specific commands and how to run them.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter:
Any error in the controller log?
Thanks,
Jun
On Tue, Sep 2, 2014 at 11:09 PM, TianYi Zhu wrote:
> Hi all,
>
> I'm running kafka 0.8.1.1, and encountering a weird problem.
> All my partitions' leader became -1 some day, and the ISR became empty, for
> example
>
> Topic:infinity PartitionCount
It seems that the producer is trying to send data for Partition [topic1,1]
to broker 3. However, that partition is hosted on broker 1. Could you try
the java client on the same broker? If that works, the issue is likely in
the Ruby producer, possibly due to incorrect metadata handling.
Thanks,
Ju
./gradlew -PscalaVersion=2.9.2 clean jar failed with JDK 8. (error: error
while loading CharSequence, class file
'/usr/java/jdk1.8.0_20/jre/lib/rt.jar(java/lang/CharSequence.class)' is
broken)
I understand there's no escape from installing JDK 7?
10x
Shlomi
On Thu, Sep 4, 2014 at 6:11 PM, Joe Ste
Hi Siddharth,
The error log shows that the kafka broker the consumer talks to is no
longer hosting the leader replica of the topic-partition, is there any
broker failures during that time (you may find it in server and controller
logs)?
Guozhang
On Thu, Sep 4, 2014 at 12:04 AM, siddharth ubale
Have you tried using a binary release http://kafka.apache.org/downloads.html
this way you don't have to-do a build?
We build using JDK 6 you should be able to run in 8 (I know for sure 6 & 7
work honestly never tried 8).
I just did a quick test with a broker running on 8 and produced/consumed a
f
I sure did. the reason I am building is trying to patch some. specifically
this : KAFKA-1623.
actually if I felt more confident about scala, I would happily send you a
patch.
If you don't care screening, just tell me how to prep it for ya and i will.
The bigger problem is running into too many open
Hello,
I am building a big web application that I want to be massively scalable (I
am using cassandra and titan as a general db).
I want to implement the following:
real time web chat that is persisted so that user a in the future can
recall his chat with user b,c,d much like facebook.
mail like
These are my rough notes on building 0.8.1
building 0.8.1
https://cwiki.apache.org/confluence/display/KAFKA/Developer+Setup
git clone https://github.com/apache/kafka.git
cd kafka
git checkout --track -b 0.8.1 origin/0.8.1
vim gradle.properties
change to: scalaVersion=2.10.1
./gradlew idea
Open
I believe there are architectures for the chat system that can use Kafka in
a sensible fashion to achieve certain of the difficult aspects. However
doing partition per user would not be advisable, nor I imagine would
relying on Kafka's storage for checking for past or expired messages. (I've
done t
I think it sounds more like another issue than your thinking...the broker
should not be failing like that especially another broker being affected
doesn't make sense.
What else is in the logs on failure? Is the other broker failing because
of number of files too? Is it happening after it becomes
No log in the recent 7 days.
kafka-preferred-replica-election.sh won't work in this stage, as the IRS is
null. I restarted the servers one by one then it's back to normal.
On Fri, Sep 5, 2014 at 1:11 AM, Jun Rao wrote:
> Any error in the controller log?
>
> Thanks,
>
> Jun
>
>
> On Tue, Sep 2,
Was there any data loss?
On Sep 4, 2014 3:52 PM, "TianYi Zhu" wrote:
> No log in the recent 7 days.
>
> kafka-preferred-replica-election.sh won't work in this stage, as the IRS is
> null. I restarted the servers one by one then it's back to normal.
>
>
> On Fri, Sep 5, 2014 at 1:11 AM, Jun Rao w
No I think, as the high level producer and consumer are still working.
On Fri, Sep 5, 2014 at 9:05 AM, Gwen Shapira wrote:
> Was there any data loss?
> On Sep 4, 2014 3:52 PM, "TianYi Zhu" wrote:
>
> > No log in the recent 7 days.
> >
> > kafka-preferred-replica-election.sh won't work in this
Thanks for looking, and confirming. The latest on bpot/poseidon resolves
this problem (but has others unfortunately), so pretty sure it's on the
client side. I'm working to patch it and get back on track for producing
from ruby apps.
On Thu, Sep 4, 2014 at 8:20 AM, Jun Rao wrote:
> It seems t
Since you want all chats and mail history persisted all the time, I
personally wouldn't recommend kafka for your requirement. Kafka is more
suitable as a streaming system where events expire after a certain time.
Look at something more general purpose like hbase for persisting data
indefinitely.
S
Agreed. I can't see this being a good use for Kafka.
Philip
-
http://www.philipotoole.com
On Thursday, September 4, 2014 9:57 PM, Sharninder wrote:
Since you want all chats and mail history persisted all the time, I
personally wouldn't recommend k
26 matches
Mail list logo
