Hello!
One of my customers wants to get the Logs produced by Arcsight into Vertica
to do some analysis.
They tried posting the normal logs to Kafka and consume from Vertica, but
Vertica doesn't understand the (binary) format and the other options
available from Arcsight are also not better for tha
Hello,
I found that we have like 1000 topics in our dev-environment because
many Devs create new topics for testing something. So I clean up the old
ones, and it was all fine. However, after restarting the cluster during
some update of the heartbeat-container, the topics are all back again.
only recent messages are ever replicated. It's possible to
configure MM2 to use an existing SMT or you can write your own as a plug-in.
Ryanne
On Thu, Dec 26, 2019, 12:25 PM Sebastian Schmitz <
sebastian.schm...@propellerhead.co.nz> wrote:
Hello Ryanne,
Is there a way to prevent t
cy to be in some
future build...
Any suggestions for this?
Thanks
Sebastian
On 30-Dec-19 1:39 PM, Sebastian Schmitz wrote:
Hello,
another thing I found and didn't find any configuration in the KIP yet
was that if I have two clusters (source and target) and a topic
"replicateme" on
t from adding the cluster-name in front of the
topic-name on target-cluster?
Thanks
Sebastian
On 27-Dec-19 7:24 AM, Sebastian Schmitz wrote:
Hello Ryanne,
Is there a way to prevent that from happening? We have two separate
clusters with some topics being replicated to the second one for
r
- MM2 will replicate the entire topic, as far back as the
retention period. However, technically there are no consumer groups in MM2!
550MB/s in a test cluster sounds pretty good to me. Try increasing
"tasks.max" and adding additional nodes.
Ryanne
On Mon, Dec 23, 2019 at 5:40 PM Se
tarted mirroring the last seven days as there was no offset
for the new consumer-group. That's next on my list to solve.
Best regards
Sebastian
On 24-Dec-19 8:34 AM, Sebastian Schmitz wrote:
Hello,
I tried running this connect-mirror-config:
name = $MIRROR_NAME
clusters = sou
s out and just wanted to know if the documentation will get
extended as the new MM2 has a lot of features and is a bit more
complicated than the old one...
Thanks
Sebastian
On 24-Dec-19 8:06 AM, Ryanne Dolan wrote:
Hello Sebastian, please let us know what issues you are facing and we can
probabl
Hello,
I'm currently trying to implement the new Kafka 2.4.0 and the new MM2.
However, it looks like the only documentation available is the KIP-382,
and the documentation
(https://kafka.apache.org/documentation/#basic_ops_mirror_maker) for the
MM isn't yet updated, and the documentation in t
Hello Kafka-Users,
I just found another problem in the 3-node-cluster I'm running on Kafka
2.3.1 with Zookeeper 3.4.14.
When running the command
./kafka-topics.sh --bootstrap-server localhost:9092 --describe
--under-replicated-partitions
the cluster told me that partition-0 of one topic has
need more of such examples to understand correct use if admin
API.
Thanks,
On Tue, 1 Oct 2019 at 19:45, Sebastian Schmitz <
sebastian.schm...@propellerhead.co.nz> wrote:
I didn't change anything with the topic... But as it only affects one
topic my idea would be that it's relate
s >1 replicas are in-sync
Kafka handles this nicely for recreating everything in the restarted broker.
I am curious, do you remember manipulating something before all these
started? e.g. Using some topic admin command (or something along the line) ?
On Tue, 1 Oct 2019 at 02:00, Sebastian S
I deleted the topic now and with topic-auto-create enabled it was
immediately recreated and all is in sync again.
Will keep and eye on this to see if it happens again
On 30-Sep-19 3:12 PM, Sebastian Schmitz wrote:
Hello again,
after like 15 minutes I have now this result:
root
same time. Partition 5 stopped on 27th,
partition 8 stopped on 25th... And so far only one topic is affected.
Thanks
Sebastian
On 30-Sep-19 2:56 PM, Sebastian Schmitz wrote:
Hello,
I just ran the kafka-reassign-partitions with --generate to create the
json and then with --execute to run it
rtition my_topic-14 completed successfully
Reassignment of partition my_topic-8 is still in progress
Reassignment of partition my_topic-4 is still in progress
Reassignment of partition my_topic-3 is still in progress
Reassignment of partition my_topic-13 completed successfully
Reassignment o
Hello,
I have one topic with 12 partitions where the partition 0 is missing one
node from ISR... Is there a way I get get it back to work again without
having to do some weird stuff like restarting the cluster?
Because this missing node in ISR is causing some problems for the
consumers...
Th
Hello,
is it possible to revert the log.message.format.version when we find out
that a consumer doesn't work with the new version?
I read in the documentation that setting this in the config would tell
Kafka that all messages in the logs are not higher than that number, so
I think we would h
Hello,
I just had a weird problem in one of my topics.
It accepted messages from producers but consumers couldn't see those
messages. The producers didn't get any errors when sending messages to
the topic, but even my test-console-consumer didn't see anything in the
topic.
Also checking the t
s/ghei36>
From: Sebastian Schmitz
Sent: Thursday, 25 July, 03:22
Subject: [EXTERNAL] Handling of inter.broker.protocol.version and
lo.message.format.version after upgrading
To: users@kafka.apache.org
Hey guys,
I wonder what I should do with both settings for future upgrades after
having f
Hey guys,
I wonder what I should do with both settings for future upgrades after
having finished the upgrade from 0.10.1 to 2.1.1.
Should I just remove it from the config so it will default to the
current version of Kafka or do I have to do the rolling upgrade with
changing versions and rest
f the zookeeper client libraries that are still
3.4.13.
I'd be interested to know if it is compatible with 3.5.x now that has a
stable release.
Mark
On Wed, 5 Jun 2019, 21:27 Sebastian Schmitz, <
sebastian.schm...@propellerhead.co.nz> wrote:
Hi,
I am currently testing the new Kafk
Hi,
I am currently testing the new Kafka 2.2.1 with new Zookeeper 3.5.5 on
my local machine to see if we can upgrade the other environments with it.
During startup of Kafka it shows Zookeeper-Version as 3.4.13:
[2019-06-05 20:16:41,486] INFO Client
environment:zookeeper.version=3.4.13-2d71af
22 matches
Mail list logo
