Yeah, looks like it's an issue with the plugin. I don't have any experience of it, sorry.
On Tue, 6 Apr. 2021, 12:32 am Himanshu Shukla, <himanshushukla...@gmail.com> wrote: > bootstrap.servers=b-1:9092,b-2:9092 > group.id=connect-cluster > key.converter=org.apache.kafka.connect.json.JsonConverter > value.converter=org.apache.kafka.connect.json.JsonConverter > key.converter.schemas.enable=true > value.converter.schemas.enable=true > offset.storage.topic=connect-offsets-2 > offset.storage.replication.factor=2 > #offset.storage.partitions=25 > config.storage.topic=connect-configs-2 > config.storage.replication.factor=2 > status.storage.topic=connect-status-2 > status.storage.replication.factor=2 > #status.storage.partitions=5 > offset.flush.interval.ms=10000 > #rest.host.name= > #rest.port=8083 > rest.advertised.host.name=172.16.234.122 > rest.advertised.port=8083 > plugin.path=/apps/libs/streamthoughts-kafka-connect-file-pulse-1.6.0/ > consumer.max.poll.records=100 > consumer.max.poll.interval.ms=600000 > > > These are the connect-distributed.properties, I am using. I have changed > the last two fields but still having the same issue. > > is it related to the source connecter(file pulse in my case). It is > scanning around 20K files and stuck. With smaller scan directory, it is > properly running. > > On Mon, Apr 5, 2021 at 2:52 PM Liam Clarke-Hutchinson < > liam.cla...@adscale.co.nz> wrote: > > > Hi Himanshu, > > > > Have you adjusted your consumer properties as the error message > suggested? > > > > Alternatively reduce your your consumer.max.poll.records in the worker > > config. > > > > Basically, the sink you're using is spending too much time processing in > > the poll loop, so either tweak the properties as mentioned in the error > > message, or reduce the number of records processed in a batch so that it > > doesn't hit that time out. > > > > If you have adjusted these properties, and still have issues, please > > respond with your current worker properties to make it easier to debug. > > > > Please note that for any KC sink or source connector, response times from > > the underlying data source/store can impact performance, so you may also > > need to look into that aspect also. > > > > Cheers, > > > > Liam Clarke-Hutchinson > > > > On Thu, 1 Apr. 2021, 5:26 pm Himanshu Shukla, < > himanshushukla...@gmail.com > > > > > wrote: > > > > > Hi, > > > I am using kafka-connect-file-pulse connector and scanning around 20K > > > files. After the scan step, the whole connect cluster is becoming > > > unresponsive. I can not even access localhost:8083/connectors/ URL. It > is > > > giving request timeout. > > > > > > > > > I have observed the below errors from the connect logs. Did anyone face > > > this issue? > > > > > > Please advise if I am doing something wrong. > > > > > > > > > [2021-03-31 16:21:58,920] INFO Scanning local file system directory > > > '/apps/datafiles_1/cm_dir/QA1/' > > > > > > > > > (io.streamthoughts.kafka.connect.filepulse.scanner.LocalFileSystemScanner:241) > > > [2021-03-31 16:22:57,586] WARN [Worker clientId=connect-1, > > > groupId=connect-cluster] This member will leave the group because > > consumer > > > poll timeout has expired. This means the time between subsequent calls > to > > > poll() was longer than the configured max.poll.interval.ms, which > > > typically > > > implies that the poll loop is spending too much time processing > messages. > > > You can address this either by increasing max.poll.interval.ms or by > > > reducing the maximum size of batches returned in poll() with > > > max.poll.records. > > > (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:1051) > > > [2021-03-31 16:22:57,586] INFO [Worker clientId=connect-1, > > > groupId=connect-cluster] *Member > > > connect-1-064cf0bf-b834-40d2-9e72-e61b229157c4 sending LeaveGroup > request > > > to coordinator URL:9092* (id: 2147483646 rack: null) > > > (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:822) > > > [2021-03-31 16:23:24,562] ERROR Request to leader to reconfigure > > connector > > > tasks failed > > > (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1037) > > > > > > > > > *org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: > > Request > > > timed out* > > > at > > > > > > > > > org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:97) > > > at > > > > > > > > > org.apache.kafka.connect.runtime.distributed.DistributedHerder$18.run(DistributedHerder.java:1034) > > > at > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > > at java.lang.Thread.run(Thread.java:748) > > > [2021-03-31 16:23:24,562] ERROR* Failed to reconfigure connector's > tasks, > > > retrying after backoff: > > > (org.apache.kafka.connect.runtime.distributed.DistributedHerder:958)* > > > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: > > Request > > > timed out > > > at > > > > > > > > > org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:97) > > > at > > > > > > > > > org.apache.kafka.connect.runtime.distributed.DistributedHerder$18.run(DistributedHerder.java:1034) > > > at > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:51 > > > > > > -- > > > Regards, > > > Himanshu Shukla > > > > > > > > -- > Regards, > Himanshu Shukla >
