[
https://issues.apache.org/jira/browse/KAFKA-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhiwei updated KAFKA-4910:
--------------------------
Description:
kafka consumer not receiving messages
consumer log：
"2017-03-10 14:35:34,448" | INFO | [Thread-5-KafkaSpout] | Revoking previously
assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
(ConsumerCoordinator.java:291)
"2017-03-10 14:35:34,451" | INFO | [Thread-5-KafkaSpout] | (Re-)joining group
data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.AbstractCoordinator
(AbstractCoordinator.java:326)
"2017-03-10 14:35:34,456" | INFO | [Thread-5-KafkaSpout] | Successfully joined
group data_storm_hw_tianlu with generation 7 |
org.apache.kafka.clients.consumer.internals.AbstractCoordinator
(AbstractCoordinator.java:434)
"2017-03-10 14:35:34,456" | INFO | [Thread-5-KafkaSpout] | Setting newly
assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
(ConsumerCoordinator.java:230)
"2017-03-10 14:36:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:36:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:37:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:37:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:38:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:38:10,297" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:39:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:39:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:40:10,298" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:40:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:40:34,454" | INFO | [Thread-5-KafkaSpout] | Revoking previously
assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
(ConsumerCoordinator.java:291)
"2017-03-10 14:40:34,457" | INFO | [Thread-5-KafkaSpout] | (Re-)joining group
data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.AbstractCoordinator
(AbstractCoordinator.java:326)
"2017-03-10 14:40:36,458" | INFO | [Thread-5-KafkaSpout] | Successfully joined
group data_storm_hw_tianlu with generation 8 |
org.apache.kafka.clients.consumer.internals.AbstractCoordinator
(AbstractCoordinator.java:434)
"2017-03-10 14:40:36,459" | INFO | [Thread-5-KafkaSpout] | Setting newly
assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
(ConsumerCoordinator.java:230)
"2017-03-10 14:41:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:41:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:42:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:42:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:43:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
kafka server log：
2017-03-10 14:39:40,767 | INFO | [kafka-scheduler-6] | Deleting index
/srv/BigData/kafka/data5/kafka-logs/BDP_RS_LINENETWORKSPEED-0/00000000000008558684.index.deleted
| kafka.log.OffsetIndex (Logging.scala:68)
2017-03-10 14:39:40,767 | INFO | [kafka-scheduler-0] | Deleting index
/srv/BigData/kafka/data2/kafka-logs/BDP_RS_BUSLINESPEED-0/00000000000000000000.index.deleted
| kafka.log.OffsetIndex (Logging.scala:68)
2017-03-10 14:40:34,457 | INFO | [kafka-request-handler-6] | [GroupCoordinator
2]: Preparing to restabilize group data_storm_hw_tianlu with old generation 7 |
kafka.coordinator.GroupCoordinator (Logging.scala:68)
2017-03-10 14:40:36,454 | INFO | [kafka-request-handler-5] | [GroupCoordinator
2]: Stabilized group data_storm_hw_tianlu generation 8 |
kafka.coordinator.GroupCoordinator (Logging.scala:68)
2017-03-10 14:40:36,455 | INFO | [kafka-request-handler-5] | [GroupCoordinator
2]: Assignment received from leader for group data_storm_hw_tianlu for
generation 8 | kafka.coordinator.GroupCoordinator (Logging.scala:68)
2017-03-10 14:41:09,459 | INFO | [executor-Heartbeat] | [GroupCoordinator 2]:
Preparing to restabilize group data_storm_hw_tianlu with old generation 8 |
kafka.coordinator.GroupCoordinator (Logging.scala:68)
2017-03-10 14:41:12,460 | INFO | [kafka-request-handler-5] | [GroupCoordinator
2]: Stabilized group data_storm_hw_tianlu generation 9 |
kafka.coordinator.GroupCoordinator (Logging.scala:68)
2017-03-10 14:41:12,462 | INFO | [kafka-request-handler-6] | [GroupCoordinator
2]: Assignment received from leader for group data_storm_hw_tianlu for
generation 9 | kafka.coordinator.GroupCoordinator (Logging.scala:68)
2017-03-10 14:41:35,758 | INFO | [kafka-request-handler-0] | Rolled new log
segment for 'MAFS_JSON_FC-0' in 1 ms. | kafka.log.Log (Logging.scala:68)
2017-03-10 14:42:19,540 | INFO | [kafka-request-handler-4] | Rolled new log
segment for '__consumer_offsets-47' in 2 ms. | kafka.log.Log (Logging.scala:68)
was:
kafka consumer not receiving messages
consumer log：
"2017-03-10 14:35:34,448" | INFO | [Thread-5-KafkaSpout] | Revoking previously
assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
(ConsumerCoordinator.java:291)
"2017-03-10 14:35:34,451" | INFO | [Thread-5-KafkaSpout] | (Re-)joining group
data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.AbstractCoordinator
(AbstractCoordinator.java:326)
"2017-03-10 14:35:34,456" | INFO | [Thread-5-KafkaSpout] | Successfully joined
group data_storm_hw_tianlu with generation 7 |
org.apache.kafka.clients.consumer.internals.AbstractCoordinator
(AbstractCoordinator.java:434)
"2017-03-10 14:35:34,456" | INFO | [Thread-5-KafkaSpout] | Setting newly
assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
(ConsumerCoordinator.java:230)
"2017-03-10 14:36:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:36:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:37:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:37:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:38:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:38:10,297" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:39:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:39:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:40:10,298" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:40:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:40:34,454" | INFO | [Thread-5-KafkaSpout] | Revoking previously
assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
(ConsumerCoordinator.java:291)
"2017-03-10 14:40:34,457" | INFO | [Thread-5-KafkaSpout] | (Re-)joining group
data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.AbstractCoordinator
(AbstractCoordinator.java:326)
"2017-03-10 14:40:36,458" | INFO | [Thread-5-KafkaSpout] | Successfully joined
group data_storm_hw_tianlu with generation 8 |
org.apache.kafka.clients.consumer.internals.AbstractCoordinator
(AbstractCoordinator.java:434)
"2017-03-10 14:40:36,459" | INFO | [Thread-5-KafkaSpout] | Setting newly
assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
(ConsumerCoordinator.java:230)
"2017-03-10 14:41:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:41:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:42:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
"2017-03-10 14:42:10,298" | INFO | [Thread-7-__system] | Getting metrics for
client connection to Netty-Client-streaming96/10.55.45.96:29106 |
backtype.storm.messaging.netty.Client (Client.java:405)
"2017-03-10 14:43:10,297" | INFO | [Thread-7-__system] | Getting metrics for
server on port 29104 | backtype.storm.messaging.netty.Server (Server.java:321)
> kafka consumer not receiving messages
> -------------------------------------
>
> Key: KAFKA-4910
> URL: https://issues.apache.org/jira/browse/KAFKA-4910
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.10.0.0
> Reporter: zhiwei
> Attachments: 128314.jstack
>
>
> kafka consumer not receiving messages
> consumer log：
> "2017-03-10 14:35:34,448" | INFO | [Thread-5-KafkaSpout] | Revoking
> previously assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group
> data_storm_hw_tianlu |
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
> (ConsumerCoordinator.java:291)
> "2017-03-10 14:35:34,451" | INFO | [Thread-5-KafkaSpout] | (Re-)joining
> group data_storm_hw_tianlu |
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> (AbstractCoordinator.java:326)
> "2017-03-10 14:35:34,456" | INFO | [Thread-5-KafkaSpout] | Successfully
> joined group data_storm_hw_tianlu with generation 7 |
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> (AbstractCoordinator.java:434)
> "2017-03-10 14:35:34,456" | INFO | [Thread-5-KafkaSpout] | Setting newly
> assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
> (ConsumerCoordinator.java:230)
> "2017-03-10 14:36:10,297" | INFO | [Thread-7-__system] | Getting metrics for
> server on port 29104 | backtype.storm.messaging.netty.Server
> (Server.java:321)
> "2017-03-10 14:36:10,298" | INFO | [Thread-7-__system] | Getting metrics for
> client connection to Netty-Client-streaming96/10.55.45.96:29106 |
> backtype.storm.messaging.netty.Client (Client.java:405)
> "2017-03-10 14:37:10,297" | INFO | [Thread-7-__system] | Getting metrics for
> server on port 29104 | backtype.storm.messaging.netty.Server
> (Server.java:321)
> "2017-03-10 14:37:10,298" | INFO | [Thread-7-__system] | Getting metrics for
> client connection to Netty-Client-streaming96/10.55.45.96:29106 |
> backtype.storm.messaging.netty.Client (Client.java:405)
> "2017-03-10 14:38:10,297" | INFO | [Thread-7-__system] | Getting metrics for
> server on port 29104 | backtype.storm.messaging.netty.Server
> (Server.java:321)
> "2017-03-10 14:38:10,297" | INFO | [Thread-7-__system] | Getting metrics for
> client connection to Netty-Client-streaming96/10.55.45.96:29106 |
> backtype.storm.messaging.netty.Client (Client.java:405)
> "2017-03-10 14:39:10,297" | INFO | [Thread-7-__system] | Getting metrics for
> server on port 29104 | backtype.storm.messaging.netty.Server
> (Server.java:321)
> "2017-03-10 14:39:10,298" | INFO | [Thread-7-__system] | Getting metrics for
> client connection to Netty-Client-streaming96/10.55.45.96:29106 |
> backtype.storm.messaging.netty.Client (Client.java:405)
> "2017-03-10 14:40:10,298" | INFO | [Thread-7-__system] | Getting metrics for
> server on port 29104 | backtype.storm.messaging.netty.Server
> (Server.java:321)
> "2017-03-10 14:40:10,298" | INFO | [Thread-7-__system] | Getting metrics for
> client connection to Netty-Client-streaming96/10.55.45.96:29106 |
> backtype.storm.messaging.netty.Client (Client.java:405)
> "2017-03-10 14:40:34,454" | INFO | [Thread-5-KafkaSpout] | Revoking
> previously assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group
> data_storm_hw_tianlu |
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
> (ConsumerCoordinator.java:291)
> "2017-03-10 14:40:34,457" | INFO | [Thread-5-KafkaSpout] | (Re-)joining
> group data_storm_hw_tianlu |
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> (AbstractCoordinator.java:326)
> "2017-03-10 14:40:36,458" | INFO | [Thread-5-KafkaSpout] | Successfully
> joined group data_storm_hw_tianlu with generation 8 |
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator
> (AbstractCoordinator.java:434)
> "2017-03-10 14:40:36,459" | INFO | [Thread-5-KafkaSpout] | Setting newly
> assigned partitions [MAFS_BPIS_ICSWIPE_IC-0] for group data_storm_hw_tianlu |
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator
> (ConsumerCoordinator.java:230)
> "2017-03-10 14:41:10,297" | INFO | [Thread-7-__system] | Getting metrics for
> server on port 29104 | backtype.storm.messaging.netty.Server
> (Server.java:321)
> "2017-03-10 14:41:10,298" | INFO | [Thread-7-__system] | Getting metrics for
> client connection to Netty-Client-streaming96/10.55.45.96:29106 |
> backtype.storm.messaging.netty.Client (Client.java:405)
> "2017-03-10 14:42:10,297" | INFO | [Thread-7-__system] | Getting metrics for
> server on port 29104 | backtype.storm.messaging.netty.Server
> (Server.java:321)
> "2017-03-10 14:42:10,298" | INFO | [Thread-7-__system] | Getting metrics for
> client connection to Netty-Client-streaming96/10.55.45.96:29106 |
> backtype.storm.messaging.netty.Client (Client.java:405)
> "2017-03-10 14:43:10,297" | INFO | [Thread-7-__system] | Getting metrics for
> server on port 29104 | backtype.storm.messaging.netty.Server
> (Server.java:321)
> kafka server log：
> 2017-03-10 14:39:40,767 | INFO | [kafka-scheduler-6] | Deleting index
> /srv/BigData/kafka/data5/kafka-logs/BDP_RS_LINENETWORKSPEED-0/00000000000008558684.index.deleted
> | kafka.log.OffsetIndex (Logging.scala:68)
> 2017-03-10 14:39:40,767 | INFO | [kafka-scheduler-0] | Deleting index
> /srv/BigData/kafka/data2/kafka-logs/BDP_RS_BUSLINESPEED-0/00000000000000000000.index.deleted
> | kafka.log.OffsetIndex (Logging.scala:68)
> 2017-03-10 14:40:34,457 | INFO | [kafka-request-handler-6] |
> [GroupCoordinator 2]: Preparing to restabilize group data_storm_hw_tianlu
> with old generation 7 | kafka.coordinator.GroupCoordinator (Logging.scala:68)
> 2017-03-10 14:40:36,454 | INFO | [kafka-request-handler-5] |
> [GroupCoordinator 2]: Stabilized group data_storm_hw_tianlu generation 8 |
> kafka.coordinator.GroupCoordinator (Logging.scala:68)
> 2017-03-10 14:40:36,455 | INFO | [kafka-request-handler-5] |
> [GroupCoordinator 2]: Assignment received from leader for group
> data_storm_hw_tianlu for generation 8 | kafka.coordinator.GroupCoordinator
> (Logging.scala:68)
> 2017-03-10 14:41:09,459 | INFO | [executor-Heartbeat] | [GroupCoordinator
> 2]: Preparing to restabilize group data_storm_hw_tianlu with old generation 8
> | kafka.coordinator.GroupCoordinator (Logging.scala:68)
> 2017-03-10 14:41:12,460 | INFO | [kafka-request-handler-5] |
> [GroupCoordinator 2]: Stabilized group data_storm_hw_tianlu generation 9 |
> kafka.coordinator.GroupCoordinator (Logging.scala:68)
> 2017-03-10 14:41:12,462 | INFO | [kafka-request-handler-6] |
> [GroupCoordinator 2]: Assignment received from leader for group
> data_storm_hw_tianlu for generation 9 | kafka.coordinator.GroupCoordinator
> (Logging.scala:68)
> 2017-03-10 14:41:35,758 | INFO | [kafka-request-handler-0] | Rolled new log
> segment for 'MAFS_JSON_FC-0' in 1 ms. | kafka.log.Log (Logging.scala:68)
> 2017-03-10 14:42:19,540 | INFO | [kafka-request-handler-4] | Rolled new log
> segment for '__consumer_offsets-47' in 2 ms. | kafka.log.Log
> (Logging.scala:68)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
