Hi Joel，
“unless you have a good reason to load balance and manage offsets manually”
In general one consumer connector consumes more than one partition.
In client side, we want to get all partitions offset for any message, if
crash happens(some message is fetched from kafka but the result is not
There are several clients available listed on the project wiki. Node.js
is among them
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+non-java+clients
Since Kafka doesn't support the websockets or HTTP directly, you would
need a middle man to redirect events from the browser to a Kafka
Thanks Jun and Joel. I've got my Kafka development instance up and running. I
do have a few questions through:
1. In /bin I note that there is kafka-server-stop.sh and
zookeeper-server-stop.sh. I assume the scripts should be executed in this order
for a clean shutdown?
2. Clearly absent are shu
During my due diligence to assess use of Kafka for both our activity and log
message streams, I would like to ask the project committers and community users
about using Kafka with Node.js. Yes, I am aware that a Kafka client exists for
Node.js (https://github.com/marcuswestin/node-kafka), which
1. Yes.
2. Producer and consumer are libraries used in client services. So, the
client app needs to call the appropriate shutdown method in the producer
and consumer library.
Thanks,
Jun
On Thu, Dec 20, 2012 at 7:28 AM, Christopher Alexander <
calexan...@gravycard.com> wrote:
> Thanks Jun and J
> Do you think this is a good reason to use SimpleConsumer rather than
> ConsumerConnector？
>
Yes, if you want to be able to rewind to some offset, SimpleConsumer is the
right API for this purpose.
On 12/20/12 11:46 AM, Christopher Alexander wrote:
During my due diligence to assess use of Kafka for both our activity and log message
streams, I would like to ask the project committers and community users about using Kafka
with Node.js. Yes, I am aware that a Kafka client exists for Node.js
Thanks David. Yes, I am aware of the Prozess Node lib also. I forgot to include
it in my posting. Good catch!
- Original Message -
From: "David Arthur"
To: users@kafka.apache.org
Sent: Thursday, December 20, 2012 11:58:45 AM
Subject: Re: Kafka Node.js Integration Questions/Advice
On 12
Chris,
Not sure how stable those node.js clients are. In 0.8, we plan to provide a
native C version of the producer. A thin node.js layer can potentially be
built on top of that.
Thanks,
Jun
On Thu, Dec 20, 2012 at 8:46 AM, Christopher Alexander <
calexan...@gravycard.com> wrote:
> During my d
“unless you have a good reason to load balance and manage offsets manually”
>
> In general one consumer connector consumes more than one partition.
> In client side, we want to get all partitions offset for any message, if
> crash happens(some message is fetched from kafka but the result is not
> f
> An alternative to using simpleconsumer in this use case is to use the
> zookeeper consumer connector and turn off auto commit.
>
Keep in mind that this works only if you don't care about controlling per
partition rewind capability.
The high level consumer will not give you control over which par
In order to support rollbacks and checkpoints, there would have to be
a way to both supply partition offsets to the consumer before reading,
as well as retrieve partition offsets from them consumer once reading
is complete.
>From what I've read here, it appears that neither the
ConsumerConnector n
> Is that the correct interpretation?
Correct.
Hi,
I have some questions about the batch size for producer.
How big should be the batch size?
What happens if producer client crashes before batch is full and messages were
still sitting in producer queue? Does it recover those messages when we
restart the producer?
Are these messages stored
> How big should be the batch size?
>
In production at Linkedin, we use a batch size of 200, which has worked
pretty well.
> What happens if producer client crashes before batch is full and messages
> were still sitting in producer queue? Does it recover those messages when
> we restart the pro
Thanks Neha.
How can I shutdown the producer? is it ctrl-c?
-Original Message-
From: Neha Narkhede [mailto:neha.narkh...@gmail.com]
Sent: Thursday, December 20, 2012 2:54 PM
To: users@kafka.apache.org
Subject: Re: Batch size in producer
> How big should be the batch size?
>
In product
Hi David,
I was looking into the listed node.js library. Prozess doesn't seem to use
zookeeper for connection.
Instead, I found one (mentioned below) which uses zookeeper based
connection in node.js .
https://npmjs.org/package/franz-kafka
https://github.com/dannycoates/franz-kafka
Are you aware
You can call the close() API on the producer
Thanks,
Neha
On Thu, Dec 20, 2012 at 3:06 PM, Subhash Agrawal wrote:
> Thanks Neha.
>
> How can I shutdown the producer? is it ctrl-c?
>
> -Original Message-
> From: Neha Narkhede [mailto:neha.narkh...@gmail.com]
> Sent: Thursday, December 20
18 matches
Mail list logo
