[
https://issues.apache.org/jira/browse/NIFI-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16281980#comment-16281980
]
Joseph Witt commented on NIFI-4676:
-----------------------------------
[~clearlove] you mention that the config is attached but it doesn't appear to
be. Please confirm.
I would like to know if this is 11+ ConsumeKafka processors all trying to talk
to the same topic or different topics and if this is a Kafka broker standalone
or a cluster of brokers. I'd like to know how many tasks each ConsumeKafka
processor is being given and how many partitions each topic has that they're
talking to.
Also, you might want to try out nifi 1.4.0 where the client version was
updated. It is possible the issues was already resolved.
> when i start consumerKafka processor more than eleven,all of the processors
> get into an indefinite stuck state
> --------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-4676
> URL: https://issues.apache.org/jira/browse/NIFI-4676
> Project: Apache NiFi
> Issue Type: Task
> Affects Versions: 1.3.0
> Environment: linux
> Reporter: clearlove
> Priority: Critical
> Attachments: 1.png, 2.png, nifi-bootstrap.log
>
>
> hello,
> When i start one consumeKafka_0_10 processor ,all of the processors are all
> right.
> but when i start consumeKafka_0_10 processor more than eleven,all of the
> processors get into an indefinite stuck state.
> !1.png|thumbnail!
> like in the picture ,look like everything is all right, but all
> processor(even processor is not consumeKafka_0_10 ) are stop work.All of the
> flowfile don't move.
> And when i stop them , some of consumeKafka_0_10 processor are warning
> {{Was interrupted while trying to communicate with Kafka with lease {}. Will
> roll back session and discard any partially received data.}}
> in the nifi-app.log
> {noformat}
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.SplitJson
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=6aacc9a9-c7e8-32b0-ed3c-3f89e72c30cf] to run
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=02aaae90-5ada-343e-6c15-3cfff7cfded0] to run
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=278dc7a5-6d2e-336a-45c3-56cb7039dc7c] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=60fd28d8-b5f0-3a5b-e31a-e3ba7b73d7cb] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=efa69212-d0c9-3c83-0985-b22115b5e861] to run
> 2017-12-05 10:12:02,724 INFO [Flow Service Tasks Thread-1]
> o.a.nifi.controller.StandardFlowService Saved flow controller
> org.apache.nifi.controller.FlowController@a987d5e // Another save pending =
> false
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=2fedc410-9641-3bda-cc40-59cf739a5400] to run
> 2017-12-05 10:12:07,130 WARN [Timer-Driven Process Thread-8]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=a2b469df-a2ef-355f-cc68-7b5dea6abb8b] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@2b5a2eed.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=d925dcdc-edef-3184-2185-47a0ebf3ea3b] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=8b6fef2d-3db8-3558-89a3-abce7b0e8797] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=e1af6798-96ba-31ec-c135-ff2f10d1306b] to run
> 2017-12-05 10:12:07,131 WARN [Timer-Driven Process Thread-7]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=d4b01bbd-62a3-3584-8391-371d52b83bce] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@7060d90d.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,132 INFO [StandardProcessScheduler Thread-8]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=b88105fe-cb1e-3c4e-f91d-22f5542cecd7] to run
> 2017-12-05 10:12:07,132 WARN [Timer-Driven Process Thread-2]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=4c067c00-fbc2-3571-5b7a-37b4957afb87] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@158837ab.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=182347e6-14ef-38c2-6873-5602f0405296] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-3]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=4035760a-8b9e-3e34-51e9-e292c58e6ed7] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4967b312.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=f7285790-6159-3726-662e-5717235a31dc] to run
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-1]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=fae14360-943d-3a81-f3b1-675f5505e265] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=86b794b6-5875-3eac-9fdb-447743459b7d] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-9]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=716f4415-e4ec-36da-8441-375bdddc690b] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@35eec0c6.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-6]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=d2e3d4a1-3257-38a0-9e7b-2e0c5c7f7395] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@261dc487.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=2fb57630-0d77-3947-954c-dc3b1fd6174e] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=4cdd0444-8047-3447-9f6c-f8f536b8228a] to run
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-1]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4f649784.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=d4372710-f550-3d31-10ae-8ae7e9300b44] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=df40050e-ebf1-368f-df61-43b5eb7bbdb6] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=8c979a96-03de-3beb-ec3a-1c91027162c9] to run
> {noformat}
> no Kerberos involved
> All configuration is default besides 'Kafka Brokers','Topic Name(s)','Group
> ID'
> The configuration file has been uploaded in the attachment
> the nifi-bootstrap.log is result of runing 'bin/nifi.sh dump'
> thanks
> Sorry,my english is so bad.Please forgive me.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
