I apologize for taking a couple days to jump in on this. We¹re currently
running JDK 1.7 u51, and we¹ve switched over to the G1 collector. If you
do this (and I highly recommend it), make sure you¹re on u51. We tried out
u21 in testing, but we had a number of problems with the GC implementation
in
Dynamically changing configs per topic is only available in 0.8.1. You can
try the latest release 0.8.1.1.
Thanks,
Jun
On Mon, May 5, 2014 at 12:28 PM, Kashyap Mhaisekar wrote:
> Thanks Jun. I am using kafka_2.8.0-0.8.0-beta1.jar. I dont see this script
> kafka-topic.sh in the bin folder. Is t
We are using Kafka for operation metrics and we do not want to loose any
data at all if there is issue with Network or all brokers needs to be
rebooted or operation reason down time while all producers are still
producing data on front end side. We use async to publish messages and we
are using Ka
Hi Neha,
How will new Consumer help us with implementing following use case?
We have heartbeat as one of topics and all application servers publish
metric to this topic. We have to meet near real-time consume SLA (less
than 30 seconds).
1) We would like to find out what is latest message per
Thanks Jun. I am using kafka_2.8.0-0.8.0-beta1.jar. I dont see this script
kafka-topic.sh in the bin folder. Is there way to do this in the version
mentioned?
--Kashyap
On Mon, May 5, 2014 at 11:18 AM, Jun Rao wrote:
> See http://kafka.apache.org/documentation.html#basic_ops_modify_topic
>
> T
Hey list,
I would like to use Kafka as an ingest point for sensitive data in a large
production network. We would like to set it up in such a way that anyone
that wants can publish data through the brokers, but we want to restrict
access for consumers to just a small subset of machines.
I'm sure
All,
Does anyone have a POM to build Java consumers and producers for Kafka? Is
there an archetype for Maven?
David Novogrodsky
david.novogrod...@gmail.com
http://www.linkedin.com/in/davidnovogrodsky
TCP will attempt to resend until the packets are successfully delivered or
a timeout occurs. So a packet loss should not lead to an error. The error
you see in your log is the queue of unsent events in the client backing up
to the point it hits its configured memory limit and then dropping events
w
Our network packet loss between the producer and kafka broker can
reach 2~5% when the traffic load is high, but in producer's log file, we
see no related error logs.
Is this expected or there should be some warnings or errors in the log file?
PS. we do see some "Event queue is full of unsent mess
See http://kafka.apache.org/documentation.html#basic_ops_modify_topic
Thanks,
Jun
On Sun, May 4, 2014 at 10:11 PM, Kashyap Mhaisekar wrote:
> Is there a way to do this at runtime using some available scripts in
> kafka/bin? If so, any pointers on which script?
>
> Regards,
> Kashyap
>
>
> On T
Yes, your understanding is correct. A global knob that controls aggregate
log size may make sense. What would be the expected behavior when that
limit is reached? Would you reduce the retention uniformly across all
topics? Then, it just means that some of the logs may not be retained as
long as you
Hi Chris,
The reason that it does not read from the beginning after the first time is
probably due to offset commit. Could you check if you set the config
auto.commit.enable
to true? Setting it to false would solve your problem that you do not need
to change to a new group.id everytime.
Guozhan
Would some one explain the interplay between the group.id and autooffset.reset
config parameters? I wrote a simple Consumer that had
group.id = "XYZZY"
autooffset.reset = smallest
My intention was to write something similar to Kafka's ConsoleConsumer; that
each time my Consumer ran, it
13 matches
Mail list logo
