Hi,
Thanks for taking this initiative forward. I'd like to propose formatting
of that table to also include current unchanged default values and
unchanged min/max constraints. In my opinion it would improve the
readability of the whole document. Having those values listed even
unchanged makes it p
Thanks Tommi.
I have again started documenting these changes in
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1030%3A+Change+constraints+and+default+values+for+various+configurations
and will try to get this KIP out of draft stage next week (so that we can
hit the 4.0 KIP freeze timeline o
Hi,
I've noticed that similar to these already mentioned settings also
segment.index.bytes has a minimum value of 4. This conflicts with
OffsetIndex, which throws `java.lang.IllegalArgumentException: Invalid max
index size: 4` with such settings, because hard-coded entry size in
OffsetIndex is 8.
Hi,
I support changing constraints for mentioned settings.
I've noticed that first producing big messages, and then setting
`segment.bytes` to low value causes unwanted consequences. I did not notice
that it would delete all the records, but I did not set it to one, but
instead the case is within
Hi all,
There are also message.max.bytes, replica.fetch.max.bytes and their
derivatives requires a constraint on their maximum value as the maximum
total memory on the instance. Otherwise, these could cause out of memory
errors on the instance.
Do you think this is in scope here as well?
On Thu,
Hi, Divij.
This isn't about config default value/constraint change though, I found
there's a behavior discrepancy in max.block.ms config, which may cause
breaking change if we change the behavior.
The detail is described in the ticket:
https://issues.apache.org/jira/browse/KAFKA-16372
What do you
One use case I see for setting the `segment.bytes` to 1 is to delete all
the records from the topic.
We can mention about it in the doc to use the `kafka-delete-records` API
instead.
On Wed, Mar 13, 2024 at 6:59 PM Divij Vaidya
wrote:
> + users@kafka
>
> Hi users of Apache Kafka
>
> With the
+ users@kafka
Hi users of Apache Kafka
With the upcoming 4.0 release, we have an opportunity to improve the
constraints and default values for various Kafka configurations.
We are soliciting your feedback and suggestions on configurations where the
default values and/or constraints should be adj
Thanks for the discussion folks. I have started a KIP
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1030%3A+Change+constraints+and+default+values+for+various+configurations
to keep track of the changes that we are discussion. Please consider this
as a collaborative work-in-progress KIP and
I think it's a great idea to raise a KIP to look at adjusting defaults and
minimum/maximum config values for version 4.0.
As pointed out, the minimum values for segment.ms and segment.bytes don't
make sense and would probably bring down a cluster pretty quickly if set
that low, so version 4.0 is a
hey guys,
Regarding to num.recovery.threads.per.data.dir: I agree, in our company we
use the number of vCPUs to do so as this is not competing with ready
cluster traffic.
On Wed, 13 Mar 2024 at 09:29, Luke Chen wrote:
> Hi Divij,
>
> Thanks for raising this.
> The valid minimum value 1 for `se
Hi Divij,
Thanks for raising this.
The valid minimum value 1 for `segment.ms` is completely unreasonable.
Similarly for `segment.bytes`, `metadata.log.segment.ms`,
`metadata.log.segment.bytes`.
In addition to that, there are also some config default values we'd like to
propose to change in v4.0.
Hey folks
Before I file a KIP to change this in 4.0, I wanted to understand the
historical context for the value of the following setting.
Currently, segment.ms minimum threshold is set to 1ms [1].
Segments are expensive. Every segment uses multiple file descriptors and
it's easy to run out of O
13 matches
Mail list logo
