Hello!
I’m struggling with a RecordTooLargeException using MirrorMaker 2 on Kafka
3.8.1 and I’d like to sanity‑check my config with the community.
Context:
-
Kafka version: 3.8.1
-
Component: MirrorSourceConnector (MirrorMaker 2)
-
Error (target side): org.apache.kafka.common.errors.RecordTooLargeException:
The message is 1049405 bytes when serialized which is larger than 1048576,
which is the value of the max.request.size configuration.
What I’m trying to do
I want MM2 to replicate messages a bit larger than 1 MB, so I added configs
along these lines:
text# In the connector / MM2 config
source->target.enabled = true
producer.max.request.size=2049405
max.request.size=2049405
target.max.request.size=2049405
At startup, I can see in the logs that the parameters are picked up with my
custom value, but after some time (under load) it behaves like it’s using
the default again and I hit the error above, which shows
max.request.size=1048576.
What I’ve checked/understood so far
-
I know that:
-
max.request.size is a producer‑side limit.
-
The default is 1 MB (1048576).
-
Topics and brokers are configured large enough (so it’s not a
message.max.bytes issue on broker/topic).
-
It looks like the MirrorSourceConnector producer is not really honoring
my overrides, or I’m not putting them in the right place / with the right
prefix.
-
*Also updated target/source broker configs* (e.g.,
message.max.bytes=2097152, socket.request.max.bytes=2097152, restarted
brokers).
Questions
1.
For MirrorMaker 2 / MirrorSourceConnector, what is the correct way to
increase the producer max.request.size?
-
Should it be:
-
producer.max.request.size in the *Connect worker* config?
-
producer.override.max.request.size in the *connector* config (as
some Strimzi examples show)?
-
Some target.* or replication.policy.* specific key?
2.
Has anyone seen the behavior where the logs at startup show the correct
custom value, but mid‑run the connector errors out still using 1048576
as max.request.size?
3.
Is there any known bug or gotcha in Kafka 3.8.x / MM2 around producer
overrides being ignored or reset for MirrorSourceConnector?
Just to clarify my setup: I’m using the connect-mirror-maker.sh script, so
MM2 is running as its own dedicated process (the embedded Connect worker),
not as a connector deployed into an external Connect cluster.
I tried enabling overrides with:
connector.client.config.override.policy=all
But it didn’t solve the issue.
>From the logs, I can see two different max.request.size values being used
by different producers:
grep -Rni max.request.size connect.log
17742:max.request.size = 2049405
18625:max.request.size = 1048576
So, it looks like one of the client instances is still using the default 1
MiB even though I’ve configured a higher value.
Given that I’m in standalone mode with connect-mirror-maker.sh, does anyone
know which exact key(s) I should use to force all MM2 producers to pick up
the larger max.request.size? For example, should I be setting something
like:
<source>.producer.max.request.size=2049405
<target>.producer.max.request.size=2049405
<source>-><target>.producer.max.request.size=2049405
