Hi All,
I understand that Kafka connect is not meant for static/batch processing
scenarios.
I am trying to accommodate static/batch load also with Kafka so that i dont
want to introduce additional layers in my architecture for doing batch.
I created a FTP connector , and it works well with small
I am running kafka in a EC2 server. While running if there is a sudden
restart of server, and then we restart the kafka broker, we can't find the
details of all the topics.
bin/kafka-topics.sh --describe --zookeeper
gives no result
bin/kafka-console-producer.sh --broker-list --topic test
yield
Hi,
I would like to process data based on a customised event time.(a timestamp that
I implement as part of the message).
The data is processed in periodic windows of x time that are parametered via
the method punctuate.
What I need is a retention time for the window to be able to treat the la
Are you restarting or terminating the instance? If your terminating, and
don't have an EBS drive attached you will lose all data on the drive.
On Fri, Oct 21, 2016 at 2:09 PM, Gourab Chowdhury
wrote:
> I am running kafka in a EC2 server. While running if there is a sudden
> restart of server, a
Hi! I see WARNs on kafka startup even if I only have a single empty topic.
Is it OK? Doesn't that mean I am going to run into real problems at some
moment?
That's the scenario, it's under Windows 10.
I download and unzip kafka 2.11 (same story with 2.10), then I make changes
to kafka-server-start
Does it stop acting as the leader (i.e. stop serving produce and fetch
requests) returning the "not a leader for partition" exception? Or
does it keep thinking it's the leader?
If it's the latter, any connected consumers that wait for new requests
on that replica will do so in vain. Since the clus
I was testing a scenario of a server crash while the Kafka is still running
and passing messages. (just did a sudo reboot). By default, I am storing
the logs at /tmp/kafka-log.
So in case an EC2 server restart, am I going to loose the data if I don't
store kafka logs in an EBS drive?
Also, I have
You are going to lose everything you store in /tmp. In a production system
you never configure Kafka or zookeeper to store critical data in /tmp.
This has nothing to do with AWS or EBS it is just standard Linux than
everything under /tmp is deleted when Linux reboots.
-hans
/**
* Hans Jespersen,
To use the new KafkaConsumer.offsetsForTimes(...) API does the server also
need to be upgraded from 0.10.0.1?
Josh
Yes, the server needs to be 0.10.1.0.
Ismael
On Sat, Oct 22, 2016 at 12:25 AM, Josh Goodrich wrote:
> To use the new KafkaConsumer.offsetsForTimes(...) API does the server also
> need to be upgraded from 0.10.0.1?
>
> Josh
>
10 matches
Mail list logo
