[
https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643249#comment-14643249
]
Guozhang Wang commented on KAFKA-2350:
--------------------------------------
[~becket_qin], I was not considering the implementation from a developer point
of view regarding "call trace", but rather from a user point of view. That is,
with two different names it is more clear about the distinction between topic /
partition subscriptions. For another example, say you write some applications
with a consumer client embedded, and very likely call its function from many
places in your app code / classes. When you saw an exception thrown from
"unsubscribe(partition)", you need to possibly look at other places and check
if it is case 1) you used subscribe(topic), but it is not assigned from Kafka,
2) you used subscribe(partition) on some other partitions, but you did not
subscribe this before. Similarly, if you saw an exception thrown on your
"subscribe(partition)" call, you need to check if 1) you called poll() in
between so that partition is no longer assigned; 2) you called subscribe(topic)
before and that partition is not one of the assigned partitions. I.e., you as
the developer needs to check what consumer function calls it has done (the
trace) before in order to trouble-shoot, when the code is not necessarily
written by yourself. With pause / resume, it will be more clear that the
consumer is certainly using topic subscriptions, and the partition is no longer
assigned to you if those functions throw an exception because of a rebalance,
etc.
> Add KafkaConsumer pause capability
> ----------------------------------
>
> Key: KAFKA-2350
> URL: https://issues.apache.org/jira/browse/KAFKA-2350
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
>
> There are some use cases in stream processing where it is helpful to be able
> to pause consumption of a topic. For example, when joining two topics, you
> may need to delay processing of one topic while you wait for the consumer of
> the other topic to catch up. The new consumer currently doesn't provide a
> nice way to do this. If you skip calls to poll() or if you unsubscribe, then
> a rebalance will be triggered and your partitions will be reassigned to
> another consumer. The desired behavior is instead that you keep the partition
> assigned and simply
> One way to achieve this would be to add two new methods to KafkaConsumer:
> {code}
> void pause(TopicPartition... partitions);
> void resume(TopicPartition... partitions);
> {code}
> Here is the expected behavior of pause/resume:
> * When a partition is paused, calls to KafkaConsumer.poll will not initiate
> any new fetches for that partition.
> * After the partition is resumed, fetches will begin again.
> * While a partition is paused, seek() and position() can still be used to
> advance or query the current position.
> * Rebalance does not preserve pause/resume state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
