Thanks, that's quite helpful. According to this post,
http://blog.empathybox.com/ , it looks like it will be beta then which
seems good enough. Assuming that the beta designation is correct, is that
because it won't have as many features/the same flexibility as it's
expected to by 0.9?
Christian
In the 0.7 docs, the description for log.retention.size and log.file.size sound
very much the same. In particular, that they apply to a single log file (or
log segment file).
http://kafka.apache.org/07/configuration.html
I'm beginning to think there is no setting to control the max aggregate s
The GC settings at http://kafka.apache.org/documentation.html#java are old.
We meant to update the documentation with the new GC settings using the G7
collector, but we haven't gotten around to doing that. Let me reach out to
our engineer, Todd Palino, who worked on tuning GC for Kafka at LinkedIn
Using Kafka 0.7.2, I have the following in server.properties:
log.retention.hours=48
log.retention.size=107374182400
log.file.size=536870912
My interpretation of this is:
a) a single log segment file over 48hrs old will be deleted
b) the total combined size of *all* logs is 100GB
c) a single log
The javadoc of the new producer is here for now -
http://empathybox.com/kafka-javadoc/index.html?kafka/clients/producer/KafkaProducer.html
Thanks,
Neha
On Thu, May 1, 2014 at 9:16 PM, Jun Rao wrote:
> The new producer (that supports callbacks) is in trunk. It will be released
> in 0.8.2. You c
The new producer (that supports callbacks) is in trunk. It will be released
in 0.8.2. You can look at the java doc of KafkaProducer for the api.
Thanks
Jun
On Thu, May 1, 2014 at 8:43 PM, Christian Csar wrote:
> On 05/01/2014 07:22 PM, Christian Csar wrote:
> > I'm looking at using the java p
Those mbean names are a bit tricky. They are created by metrics-code and
have quotes in them. So, you will need to be a bit careful.
Thanks,
Jun
On Thu, May 1, 2014 at 12:39 PM, Priya wrote:
> I am integrating the Kafka JMX metrics into our own system. I am able to
> view the Kafka metrics vi
http://kafka.apache.org/documentation.html#java
Thanks,
Jun
On Thu, May 1, 2014 at 12:19 PM, Cassa L wrote:
> Hi,
> I want to know what usually, are the JAVA_HEAP settings recommended for
> kafka servers in production.
>
> Thanks
> LCassa
>
On 05/01/2014 07:22 PM, Christian Csar wrote:
> I'm looking at using the java producer api for 0.8.1 and I'm slightly
> confused by this passage from section 4.4 of
> https://kafka.apache.org/documentation.html#theproducer
> "Note that as of Kafka 0.8.1 the async producer does not have a
> callback
I'm looking at using the java producer api for 0.8.1 and I'm slightly
confused by this passage from section 4.4 of
https://kafka.apache.org/documentation.html#theproducer
"Note that as of Kafka 0.8.1 the async producer does not have a
callback, which could be used to register handlers to catch send
Hi Edward,
What is the current version you are using? Starting in 0.8.1 the default
partitioner of the producer would no longer be round-robin at a per-message
level but would stick to one partition for a period (the metadata refresh
period, which is 10 min by default) and then round-robin to anot
Hi Ian,
That is correct. The consumer fetcher threads get data from multiple
partitions in batches, and put batches in to the stream one-by-one, hence
when stream iterates, it will likely give a sequence of messages from one
partition, then from another one, and so on.
Guozhang
On Thu, May 1, 2
Hey gang, quick consumer behavior question:
If I have a Consumer that owns several partitions but only creates a single
stream for them, in what order will that stream get its messages? From my
experiments it seems like the Consumer fetches some amount of messages for all
N partitions it holds,
After installing Kafka I did the following
createTopic --partitions 10 -replication 1 --name mytopic
I started producer using bin/kafka-producer-shell
I opened two consumers in the same group using bin/kafka-consumer-shell
In older versions this test used to result in the messages being sent by
David - We're currently running 3.4.6 without issues, though our load is
modest.
Clark
On Thu, May 1, 2014 at 2:26 PM, Neha Narkhede wrote:
> Through our experience of operating zookeeper in production at LinkedIn, we
> found 3.3.4 to be very stable. It is likely at 3.4.x is stable now, but we
>
Through our experience of operating zookeeper in production at LinkedIn, we
found 3.3.4 to be very stable. It is likely at 3.4.x is stable now, but we
can't say for sure as we haven't had a chance to deploy 3.4.x in production
yet.
Thanks,
Neha
On Thu, May 1, 2014 at 12:13 PM, David Peterson wr
Hello all,
After updating to latest 0.8.1.1 jmxtrans fails with Connection timeout error.
I can connect with jconsole, but it is very unstable, reconnects every
few seconds and sometimes fails with
May 01, 2014 2:03:13 PM ClientCommunicatorAdmin Checker-run
WARNING: Failed to check connection: jav
I am integrating the Kafka JMX metrics into our own system. I am able to view
the Kafka metrics via JConsole. However, pulling the metrics into my own
system, using my JSON files is failing. It is unable to find the mbeans
("javax.management.InstanceNotFoundException:
kafka.server:type=BrokerTo
Hi,
I want to know what usually, are the JAVA_HEAP settings recommended for
kafka servers in production.
Thanks
LCassa
Is there any compelling reason to use the latest stable ZooKeeper instead of
the 3.3.4 version included with the Kafka distribution?
We’re not using ZooKeeper for any other application - we’re only running it
because it’s a Kafka dependency.
Thank you,
-dave.
Hi David,
We have merged all the topic related tools in kafka-topics.sh in Linux, if
it is missing a correspondence .bat file then this is a bug. If so could
you file a JIRA?
Guozhang
On Thu, May 1, 2014 at 7:21 AM, David Novogrodsky <
david.novogrod...@gmail.com> wrote:
> Hello,
>
> I am expe
Hello,
I am experimenting with Kafka. The binary version of kafka_2.9.2-0.8.1.1.
I am running it on a Windows system. I am trying to run the scripts in
the windows directory to make a topic and make some sample messages. Here
is the error I am getting:
--
C:\kafka_2.9.2-0.8.1.1>.\bin\windows\k
Yes, it helps a lot. When I run in async mode my producer stays up and healthy
even if zookeeper and/or Kafka brokers are down.
Thanks,
C. Helck
-Original Message-
From: Dragos Manolescu [mailto:dmanole...@whitepages.com]
Sent: Wednesday, April 30, 2014 5:50 PM
To: users@kafka.apache.
Jun,
Thanks for the reply, I appreciate it.
David Novogrodsky
david.novogrod...@gmail.com
http://www.linkedin.com/in/davidnovogrodsky
On Wed, Apr 30, 2014 at 11:33 PM, Jun Rao wrote:
> We fixed the windows scripts in trunk. However, we didn't backport it to
> 0.8.1.1. Could you try the window
24 matches
Mail list logo
