[
https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14636098#comment-14636098
]
Jay Kreps commented on KAFKA-2350:
----------------------------------
+1 on resume instead of unpause though it doesn't match subscribe/unsubscribe.
The original motivation for this was to be able to subscribe at the topic level
but be able to say that while you're still subscribed to a given partition you
can't take more data for that partition at this particular moment. Generalizing
that to allow pausing a whole topic makes sense too.
[~becket_qin] I think your idea is having unsubscribe(partition) have the same
effect as pause(partition) when you are subscribed at the topic level would be
intuitive, but the logic of how that would work might be a bit complex. If
someone is smart enough to work out the details that could be more elegant than
a new api. The challenge is that partition level subscribe/unsubscribe is
currently an error if you are subscribed at the topic level and the details of
that control whether group management etc is used too.
> Add KafkaConsumer pause capability
> ----------------------------------
>
> Key: KAFKA-2350
> URL: https://issues.apache.org/jira/browse/KAFKA-2350
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
>
> There are some use cases in stream processing where it is helpful to be able
> to pause consumption of a topic. For example, when joining two topics, you
> may need to delay processing of one topic while you wait for the consumer of
> the other topic to catch up. The new consumer currently doesn't provide a
> nice way to do this. If you skip poll() or if you unsubscribe, then a
> rebalance will be triggered and your partitions will be reassigned.
> One way to achieve this would be to add two new methods to KafkaConsumer:
> {code}
> void pause(String... topics);
> void unpause(String... topics);
> {code}
> When a topic is paused, a call to KafkaConsumer.poll will not initiate any
> new fetches for that topic. After it is unpaused, fetches will begin again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
