Hello,
I'm having trouble getting Kafka to compact a topic. It's over 300GB and
has enough segments to warrant cleaning. It should only be about 40 GB
(there is a copy in a db that is unique on the key). Below are the
configs we have (default broker) and topic override.
Is there something I
On 10/25/2017 12:03 PM, Manikumar wrote:
any errors in log cleaner logs?
Not as far as I can see
root@kafka-1:/opt/kafka/logs# cat log-cleaner.log* | grep -i error
(empty)
However, I've seen that it actually did cleaning of the whole topic
(excerpts below), but it didn't seem to find anythin
Hi,
On 10/25/2017 12:15 PM, Xin Li wrote:
> I think that is a bug, and should be fixed in this task
https://issues.apache.org/jira/browse/KAFKA-6030.
> We experience that in our kafka cluster, we just check out the 11.0.2
version, build it ourselves.
thanks for the hint, as it looks like a c
ll small.
With upgrading you could also just cherrypick this very commit or put a
logstatement to verify.
Given the Logsizes your dealing with, I am very confident that this is
your issue.
Best Jan
On 25.10.2017 12:21, Elmar Weber wrote:
Hi,
On 10/25/2017 12:15 PM, Xin Li wrote:
> I think
rf
* This email message may contain legally privileged and/or confidential
information.
You are hereby notified that any disclosure, copying, distribution, or use of
this email message is strictly prohibited.
On 25.10.17, 12:34, "Elmar Weber" wrote:
Hi,
thanks, I'll
Hello,
I had this morning the issue that a client offset got deleted from a
broker as it seems.
(Kafka 0.11.0.1 with patch for KAFKA-6030 on top)
It happened in a test environment and the pipeline stage got re-deployed
multiple times, during one restart, when the consumer reconnected, it
di
go through. Consider using 3rd party site.
On Wed, Nov 1, 2017 at 9:18 AM, Elmar Weber wrote:
Hello,
I had this morning the issue that a client offset got deleted from a
broker as it seems.
(Kafka 0.11.0.1 with patch for KAFKA-6030 on top)
It happened in a test environment and the pipeline st
