[ https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Gustafson updated KAFKA-4254: ----------------------------------- Comment: was deleted (was: [~ijuma] I'm sympathetic to your concern, but it seems a little difficult to justify the current behavior of raising an exception based on stale metadata. As a user, I expect that I can produce to an existing partition even if it has just been created, so requiring me to catch exceptions until the producer's metadata refresh interval expires seems a bit clunky. Of course, the producer doesn't necessarily need to block the full {{max.block.ms}} until the partition is created, maybe one metadata refresh prior to raising would be sufficient?) > Questionable handling of unknown partitions in KafkaProducer > ------------------------------------------------------------ > > Key: KAFKA-4254 > URL: https://issues.apache.org/jira/browse/KAFKA-4254 > Project: Kafka > Issue Type: Bug > Components: producer > Reporter: Jason Gustafson > Assignee: Jason Gustafson > Fix For: 0.10.1.1 > > > Currently the producer will raise an {{IllegalArgumentException}} if the user > attempts to write to a partition which has just been created. This is caused > by the fact that the producer does not attempt to refetch topic metadata in > this case, which means that its check for partition validity is based on > stale metadata. > If the topic for the partition did not already exist, it works fine because > the producer will block until it has metadata for the topic, so this case is > primarily hit when the number of partitions is dynamically increased. > A couple options to fix this that come to mind: > 1. We could treat unknown partitions just as we do unknown topics. If the > partition doesn't exist, we refetch metadata and try again (timing out when > max.block.ms is reached). > 2. We can at least throw a more specific exception so that users can handle > the error. Raising {{IllegalArgumentException}} is not helpful in practice > because it can also be caused by other error.s > My inclination is to do the first one since the producer seems incorrect to > tell the user that the partition is invalid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)