[
https://issues.apache.org/jira/browse/KAFKA-3503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navin Markandeya updated KAFKA-3503:
------------------------------------
Description:
I would expect some exception to be thrown when a consumer tries to access a
non-existent partition. I did not see anyone reporting it. If is already known,
please link and close this.
{code}
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
{code}
{code}
Linux centos65vm 2.6.32-573.el6.x86_64 #1 SMP Thu Jul 23 15:44:03 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux
{code}
{{Kafka release - kafka_2.11-0.9.0.1}}
Created a topic with 3 partitions
{code}
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic mytopic
Topic:mytopic PartitionCount:3 ReplicationFactor:1 Configs:
Topic: mytopic Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: mytopic Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: mytopic Partition: 2 Leader: 0 Replicas: 0 Isr: 0
{code}
Consumer application does not terminate. A thrown exception that there is no
such {{mytopic-3}} partition, that would help to gracefully terminate it.
{code}
14:08:02.885 [main] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Fetching committed
offsets for partitions: [mytopic-3, mytopic-0, mytopic-1, mytopic-2]
14:08:02.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with
name node-2147483647.bytes-sent
14:08:02.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with
name node-2147483647.bytes-received
14:08:02.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with
name node-2147483647.latency
14:08:02.888 [main] DEBUG o.apache.kafka.clients.NetworkClient - Completed
connection to node 2147483647
14:08:02.891 [main] DEBUG o.a.k.c.c.i.ConsumerCoordinator - No committed offset
for partition mytopic-3
14:08:02.891 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Resetting offset
for partition mytopic-3 to latest offset.
14:08:02.892 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
mytopic-3 is unknown for fetching offset, wait for metadata refresh
14:08:02.965 [main] DEBUG o.apache.kafka.clients.NetworkClient - Sending
metadata request ClientRequest(expectResponse=true, callback=null,
request=RequestSend(header={api_key=3,api_version=0,correlation_id=4,client_id=consumer-2},
body={topics=[mytopic]}), isInitiatedByNetworkClient,
createdTimeMs=1459804082965, sendTimeMs=0) to node 0
14:08:02.968 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster
metadata version 3 to Cluster(nodes = [Node(0, centos65vm, 9092)], partitions =
[Partition(topic = mytopic, partition = 0, leader = 0, replicas = [0,], isr =
[0,], Partition(topic = mytopic, partition = 1, leader = 0, replicas = [0,],
isr = [0,], Partition(topic = mytopic, partition = 2, leader = 0, replicas =
[0,], isr = [0,]])
14:08:02.968 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
mytopic-3 is unknown for fetching offset, wait for metadata refresh
14:08:03.071 [main] DEBUG o.apache.kafka.clients.NetworkClient - Sending
metadata request ClientRequest(expectResponse=true, callback=null,
request=RequestSend(header={api_key=3,api_version=0,correlation_id=5,client_id=consumer-2},
body={topics=[mytopic]}), isInitiatedByNetworkClient,
createdTimeMs=1459804083071, sendTimeMs=0) to node 0
14:08:03.073 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster
metadata version 4 to Cluster(nodes = [Node(0, centos65vm, 9092)], partitions =
[Partition(topic = mytopic, partition = 0, leader = 0, replicas = [0,], isr =
[0,], Partition(topic = mytopic, partition = 1, leader = 0, replicas = [0,],
isr = [0,], Partition(topic = mytopic, partition = 2, leader = 0, replicas =
[0,], isr = [0,]])
14:08:03.073 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
mytopic-3 is unknown for fetching offset, wait for metadata refresh
{code}
was:
I would expect some exception to be thrown when a consumer tries to access a
non-existent partition. I did not see anyone reporting it. If is already known,
please link and close this.
{code}
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
{code}
{code}
Linux centos65vm 2.6.32-573.el6.x86_64 #1 SMP Thu Jul 23 15:44:03 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux
{code}
{{Kafka release - kafka_2.11-0.9.0.1}}
Created a topic with 3 partitions
{code}
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic mytopic
Topic:mytopic PartitionCount:3 ReplicationFactor:1 Configs:
Topic: mytopic Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: mytopic Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: mytopic Partition: 2 Leader: 0 Replicas: 0 Isr: 0
{code}
Consumer application does not terminate. A thrown exception would help to
gracefully terminate it.
{code}
14:08:02.885 [main] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Fetching committed
offsets for partitions: [mytopic-3, mytopic-0, mytopic-1, mytopic-2]
14:08:02.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with
name node-2147483647.bytes-sent
14:08:02.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with
name node-2147483647.bytes-received
14:08:02.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with
name node-2147483647.latency
14:08:02.888 [main] DEBUG o.apache.kafka.clients.NetworkClient - Completed
connection to node 2147483647
14:08:02.891 [main] DEBUG o.a.k.c.c.i.ConsumerCoordinator - No committed offset
for partition mytopic-3
14:08:02.891 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Resetting offset
for partition mytopic-3 to latest offset.
14:08:02.892 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
mytopic-3 is unknown for fetching offset, wait for metadata refresh
14:08:02.965 [main] DEBUG o.apache.kafka.clients.NetworkClient - Sending
metadata request ClientRequest(expectResponse=true, callback=null,
request=RequestSend(header={api_key=3,api_version=0,correlation_id=4,client_id=consumer-2},
body={topics=[mytopic]}), isInitiatedByNetworkClient,
createdTimeMs=1459804082965, sendTimeMs=0) to node 0
14:08:02.968 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster
metadata version 3 to Cluster(nodes = [Node(0, centos65vm, 9092)], partitions =
[Partition(topic = mytopic, partition = 0, leader = 0, replicas = [0,], isr =
[0,], Partition(topic = mytopic, partition = 1, leader = 0, replicas = [0,],
isr = [0,], Partition(topic = mytopic, partition = 2, leader = 0, replicas =
[0,], isr = [0,]])
14:08:02.968 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
mytopic-3 is unknown for fetching offset, wait for metadata refresh
14:08:03.071 [main] DEBUG o.apache.kafka.clients.NetworkClient - Sending
metadata request ClientRequest(expectResponse=true, callback=null,
request=RequestSend(header={api_key=3,api_version=0,correlation_id=5,client_id=consumer-2},
body={topics=[mytopic]}), isInitiatedByNetworkClient,
createdTimeMs=1459804083071, sendTimeMs=0) to node 0
14:08:03.073 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster
metadata version 4 to Cluster(nodes = [Node(0, centos65vm, 9092)], partitions =
[Partition(topic = mytopic, partition = 0, leader = 0, replicas = [0,], isr =
[0,], Partition(topic = mytopic, partition = 1, leader = 0, replicas = [0,],
isr = [0,], Partition(topic = mytopic, partition = 2, leader = 0, replicas =
[0,], isr = [0,]])
14:08:03.073 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
mytopic-3 is unknown for fetching offset, wait for metadata refresh
{code}
> Throw exception on missing/non-existent partition
> ---------------------------------------------------
>
> Key: KAFKA-3503
> URL: https://issues.apache.org/jira/browse/KAFKA-3503
> Project: Kafka
> Issue Type: Wish
> Affects Versions: 0.9.0.1
> Environment: Java 1.8.0_60.
> Linux centos65vm 2.6.32-573.el6.x86_64 #1 SMP Thu Jul 23 15:44:03 UTC
> Reporter: Navin Markandeya
> Priority: Minor
>
> I would expect some exception to be thrown when a consumer tries to access a
> non-existent partition. I did not see anyone reporting it. If is already
> known, please link and close this.
> {code}
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> {code}
> {code}
> Linux centos65vm 2.6.32-573.el6.x86_64 #1 SMP Thu Jul 23 15:44:03 UTC 2015
> x86_64 x86_64 x86_64 GNU/Linux
> {code}
> {{Kafka release - kafka_2.11-0.9.0.1}}
> Created a topic with 3 partitions
> {code}
> bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic mytopic
> Topic:mytopic PartitionCount:3 ReplicationFactor:1 Configs:
> Topic: mytopic Partition: 0 Leader: 0 Replicas: 0 Isr: 0
> Topic: mytopic Partition: 1 Leader: 0 Replicas: 0 Isr: 0
> Topic: mytopic Partition: 2 Leader: 0 Replicas: 0 Isr: 0
> {code}
> Consumer application does not terminate. A thrown exception that there is no
> such {{mytopic-3}} partition, that would help to gracefully terminate it.
> {code}
> 14:08:02.885 [main] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Fetching
> committed offsets for partitions: [mytopic-3, mytopic-0, mytopic-1, mytopic-2]
> 14:08:02.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor
> with name node-2147483647.bytes-sent
> 14:08:02.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor
> with name node-2147483647.bytes-received
> 14:08:02.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor
> with name node-2147483647.latency
> 14:08:02.888 [main] DEBUG o.apache.kafka.clients.NetworkClient - Completed
> connection to node 2147483647
> 14:08:02.891 [main] DEBUG o.a.k.c.c.i.ConsumerCoordinator - No committed
> offset for partition mytopic-3
> 14:08:02.891 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Resetting
> offset for partition mytopic-3 to latest offset.
> 14:08:02.892 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
> mytopic-3 is unknown for fetching offset, wait for metadata refresh
> 14:08:02.965 [main] DEBUG o.apache.kafka.clients.NetworkClient - Sending
> metadata request ClientRequest(expectResponse=true, callback=null,
> request=RequestSend(header={api_key=3,api_version=0,correlation_id=4,client_id=consumer-2},
> body={topics=[mytopic]}), isInitiatedByNetworkClient,
> createdTimeMs=1459804082965, sendTimeMs=0) to node 0
> 14:08:02.968 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster
> metadata version 3 to Cluster(nodes = [Node(0, centos65vm, 9092)], partitions
> = [Partition(topic = mytopic, partition = 0, leader = 0, replicas = [0,], isr
> = [0,], Partition(topic = mytopic, partition = 1, leader = 0, replicas =
> [0,], isr = [0,], Partition(topic = mytopic, partition = 2, leader = 0,
> replicas = [0,], isr = [0,]])
> 14:08:02.968 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
> mytopic-3 is unknown for fetching offset, wait for metadata refresh
> 14:08:03.071 [main] DEBUG o.apache.kafka.clients.NetworkClient - Sending
> metadata request ClientRequest(expectResponse=true, callback=null,
> request=RequestSend(header={api_key=3,api_version=0,correlation_id=5,client_id=consumer-2},
> body={topics=[mytopic]}), isInitiatedByNetworkClient,
> createdTimeMs=1459804083071, sendTimeMs=0) to node 0
> 14:08:03.073 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster
> metadata version 4 to Cluster(nodes = [Node(0, centos65vm, 9092)], partitions
> = [Partition(topic = mytopic, partition = 0, leader = 0, replicas = [0,], isr
> = [0,], Partition(topic = mytopic, partition = 1, leader = 0, replicas =
> [0,], isr = [0,], Partition(topic = mytopic, partition = 2, leader = 0,
> replicas = [0,], isr = [0,]])
> 14:08:03.073 [main] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition
> mytopic-3 is unknown for fetching offset, wait for metadata refresh
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
