You probably have the wrong version of the Kafka jar(s) within your
classpath. Which version of Kafka are you using and how have you setup
the classpath?
-Jaikiran
On Thursday 18 June 2015 08:11 AM, Srividhya Anantharamakrishnan wrote:
Hi,
I am trying to set up Kafka in our cluster and I am r
@Shapira You are correct from my perspective. We are using kafka for a
system where panels can send multiple events in a single message. The
current contract is such that all events fail or succeed as a whole. If
there is a failure the panel resends all the events. The existing producer
api support
Hi,
I am trying to set up Kafka in our cluster and I am running into the
following error when Consumer is getting instantiated:
java.lang.NoSuchMethodError:
org.apache.kafka.common.utils.Utils.newThread(Ljava/lang/String;Ljava/lang/Runnable;Ljava/lang/Boolean;)Ljava/lang/Thread;
at
kafka
3 Node Kafka - 0.8.2.1
3 node ZK - 3.4.6
We experienced a soft-node failure from one of our brokers (#2). The
process was still running but no logs were being generated, it was not
responding to JMX queries etc.
Several consumers were unable to read from certain partitions while this
was occurri
We have a couple open tickets to address these issues (see KAFKA-1894 and
KAFKA-2168). It's definitely something we want to fix.
On Wed, Jun 17, 2015 at 4:21 AM, Jan Stette wrote:
> Adding some more details to the previous question:
>
> The indefinite wait doesn't happen on calling subscribe() o
Hi Carl,
"** Disclaimer: I know there's a new consumer API on the way, this mail is
about the currently available API. I also apologise if the below has
already been discussed previously. I did try to check previous discussions
It seems to me that the high-level consumer would be able to support
a
I have configured my log4j with Kafka Appender.(Kafka version 0.8.2.0)
Following are entries from my log4j file
log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
log4j.appender.KAFKA.BrokerList=iakafka301p.dev.ch3.s.com:9092,
iakafka302p.dev.ch3.s.com:9092,iakafka303p.dev.ch3.s.com:9092
log4j
+1 to using exhibitor. Besides managing the ensemble it also helps with
backups and zk log cleanup (which if you don't do your machine will run out
of space).
~ Joestein
On Jun 17, 2015 9:44 AM, "Dillian Murphey" wrote:
> supervisord is pretty easy to use. Netflix Exhibitor will manage this all
supervisord is pretty easy to use. Netflix Exhibitor will manage this all
for zookeeper, if you want to try that tool.
On Wed, Jun 17, 2015 at 7:03 AM, Kashyap Mhaisekar
wrote:
> We use supervisord for this. It ensures that the processes are always up
> and running.
>
> Thanks
> Kashyap
>
> On
I set this up on EC2 in exactly the same way and had the same errors when
accessing it with a producer that was outside EC2. Is there something else
I have to configure other than to set "advertised.host.name" to my external
IP address?
On Tue, Jun 16, 2015 at 4:27 PM, Mike Bridge wrote:
> Run
This is actually an expected consequence of using distributed systems. The
kafka FAQ has a good answer
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowdoIgetexactly-oncemessagingfromKafka
?
On Tue, Jun 16, 2015 at 11:06 PM, Kris K wrote:
> Hi,
>
> While testing message delivery usi
We use supervisord for this. It ensures that the processes are always up
and running.
Thanks
Kashyap
On Wednesday, June 17, 2015, Shayne S wrote:
> kafka-server-start.sh has a -daemon option, but I don't think Zookeeper has
> it.
>
> On Tue, Jun 16, 2015 at 11:32 PM, Su She > wrote:
>
> > It s
Ah misread that sorry!
On 17.06.2015 14:26, Shayne S wrote:
Right, you can see I've got segment.ms set. The trick is that they don't
actually roll over until something new arrives. If your topic is idle (not
receiving messages), it won't ever roll over to a new segment, and thus the
last segmen
kafka-server-start.sh has a -daemon option, but I don't think Zookeeper has
it.
On Tue, Jun 16, 2015 at 11:32 PM, Su She wrote:
> It seems like nohup has solved this issue, even when the putty window
> becomes inactive the processes are still running (I din't need to
> interact with them). I mig
Right, you can see I've got segment.ms set. The trick is that they don't
actually roll over until something new arrives. If your topic is idle (not
receiving messages), it won't ever roll over to a new segment, and thus the
last segment will never be compacted.
Thanks!
Shayne
On Wed, Jun 17, 201
Adding some more details to the previous question:
The indefinite wait doesn't happen on calling subscribe() on the consumer,
it happens when I (in this case) call seekToEnd().
A related problem to this is that the seekToEnd() method is synchronized
(as are the other access methods on KafkaConsum
Hi,
you might want to have a look here:
http://kafka.apache.org/documentation.html#topic-config
_segment.ms_ and _segment.bytes _ should allow you to control the
time/size when segments are rolled.
Best
Jan
On 16.06.2015 14:05, Shayne S wrote:
Some further information, and is this a bug? I
I'm trying out the new KafkaConsumer client API in the trunk of the source
tree, and while I realise that this is a work in progress, I have a
question that perhaps someone can shed some light on.
I'm looking at how to handle various error scenarios for a Kafka client, in
particular what happens w
So Carl Heymann's ConsumerIterator.next hack approach is not reasonable?
2015-06-17 08:12:50 + 上のメッセージ Stevo Slavić:
--047d7bfcf30ed09b460518b241db
Content-Type: text/plain; charset=UTF-8
With auto-commit one can only have at-most-once delivery guarantee - after
commit but before messa
Most of the tuning options are available in kafka-run-class.sh. You can
override required props (KAFKA_HEAP_OPTS , KAFKA_JVM_PERFORMANCE_OPTS) to
kafka-server-start.sh script.
On Wed, Jun 17, 2015 at 2:11 PM, luo.fucong wrote:
> I want to tune the kafka jvm options, but nowhere can I pass the op
I want to tune the kafka jvm options, but nowhere can I pass the options to the
kafka startup script(bin/kafka-server-start.sh).
How to pass in the jvm options?
With auto-commit one can only have at-most-once delivery guarantee - after
commit but before message is delivered for processing, or even after it is
delivered but before it is processed, things can fail, causing event not to
be processed, which is basically same outcome as if it was not delivered.
Thank you for the reply.
Patch submitted https://issues.apache.org/jira/browse/KAFKA-2281
On Mon, 15 Jun 2015 at 02:16 Jiangjie Qin wrote:
> Hi Tao,
>
> Yes, the issue that ErrorLoggingCallback keeps value as local variable is
> known for a while and we probably should fix it as the value is no
23 matches
Mail list logo
