Hi, you need to increase record and message size because your real message
payload is bigger than what’s mention in properties file.
Regards,
On Fri, 1 Jul 2022 at 20:24, Divya Jain <truedeeds.divya1...@gmail.com>
wrote:
> Hi,
>
> I am facing this issue:
> 2022-07-01 19:01:05,548] INFO Topic 'postgres.public.content_history'
> already exists. (org.apache.kafka.connect.runtime.WorkerSourceTask:423)
> [2022-07-01 19:01:05,641] INFO
> WorkerSourceTask{id=smtip-de-content2-source-connector-0} Committing
> offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:488)
> [2022-07-01 19:01:05,642] INFO
> WorkerSourceTask{id=smtip-de-content2-source-connector-0} flushing 155
> outstanding messages for offset commit
> (org.apache.kafka.connect.runtime.WorkerSourceTask:505)
> [2022-07-01 19:01:06,034] INFO
> WorkerSourceTask{id=smtip-de-content2-source-connector-0} Finished
> commitOffsets successfully in 393 ms
> (org.apache.kafka.connect.runtime.WorkerSourceTask:586)
> [2022-07-01 19:01:06,575] WARN [Producer
> clientId=connector-producer-smtip-de-content2-source-connector-0] Got error
> produce response in correlation id 610 on topic-partition
> postgres.public.content_history-1, splitting and retrying (2147483647
> attempts left). Error: MESSAGE_TOO_LARGE
> (org.apache.kafka.clients.producer.internals.Sender:582)
> [2022-07-01 19:01:06,843] ERROR
> WorkerSourceTask{id=smtip-de-content2-source-connector-0} failed to send
> record to postgres.public.content_history:
> (org.apache.kafka.connect.runtime.WorkerSourceTask:372)
> org.apache.kafka.common.errors.RecordTooLargeException: The request
> included a message larger than the max message size the server will accept.
> [2022-07-01 19:01:06,927] WARN [Producer
> clientId=connector-producer-smtip-de-content2-source-connector-0] Got error
> produce response in correlation id 643 on topic-partition
> postgres.public.content_history-0, splitting and retrying (2147483647
> attempts left). Error: MESSAGE_TOO_LARGE
> (org.apache.kafka.clients.producer.internals.Sender:582)
> [2022-07-01 19:01:06,941] INFO The task will send records to topic
> 'postgres.public.content_attributes' for the first time. Checking whether
> topic exists (org.apache.kafka.connect.runtime.WorkerSourceTask:419)
> [2022-07-01 19:01:06,943] INFO Topic 'postgres.public.content_attributes'
> already exists. (org.apache.kafka.connect.runtime.WorkerSourceTask:423)
>
>
> The properties I am using in my worker.properties are: I have tried
> combination of properties I am not sure how to fix it. Please guide on the
> same.
>
> offset.flush.timeout.ms=60000
>
> offset.flush.interval.ms=5000
>
> max.partition.fetch.bytes=2147483647
>
> max.request.size=2147483647
>
> max.message.bytes=2147483647
>
> message.max.bytes=2147483647
>
> replica.fetch.max.bytes=2147483647
>
> producer.security.protocol=SSL
>
> producer.max.request.size=2147483647
>
> connector.client.config.override.policy=All
>
> Thanks & Regards
> Divya Jain
>
