I was planning to start a new thread by my experience is similar to the one
described here. I am currently using 0.8.2-beta and my setup has a producer
producing to topic "DC-DATA", kafka brokers with auto-topic-creation
enabled, and mirrormaker set to replicate any ".*-DATA" topics. After the
prod
Hmm.. that's right. completely forgot about that.
On Mon, Jan 26, 2015 at 01:49:33PM -0800, Jun Rao wrote:
> Joel,
>
> That's probably because console consumer always uses wildcard for
> consumption.
>
> Thanks,
>
> Jun
>
> On Mon, Jan 26, 2015 at 1:44 PM, Joel Koshy wrote:
>
> > I think thi
Joel,
That's probably because console consumer always uses wildcard for
consumption.
Thanks,
Jun
On Mon, Jan 26, 2015 at 1:44 PM, Joel Koshy wrote:
> I think this should work even on non-wildcard consumers. (i.e., I just
> tried it locally and it appears to work fine).
>
> Here is what I did:
I think this should work even on non-wildcard consumers. (i.e., I just
tried it locally and it appears to work fine).
Here is what I did:
- Leave a console consumer up and running, bounce the broker to turn
off auto-create
- Delete the topic
- The consumer should rebalance and stop consuming th
Actually, I think the deletion of a topic will trigger a rebalance on all
wildcard subscribers. The time to complete the rebalance depends on the #
of topic/partitions.
Thanks,
Jun
On Mon, Jan 26, 2015 at 12:26 PM, Jason Rosenberg wrote:
> Yeah Joel,
>
> I just thought of that idea too (e.g. t
Yeah Jun,
I realize it's not worth blocking things, since it's not trivial to solve.
Thanks for all the work on this.
Jason
On Mon, Jan 26, 2015 at 3:01 PM, Jun Rao wrote:
> Hi, Jason,
>
> I am not sure that we should delay the 0.8.2 release. The reasons are (1)
> There are other features suc
Yeah Joel,
I just thought of that idea too (e.g. temporarily disable auto topic
creation). I can probably make that work (it may result in unexpected lack
of topic creation, but I can message this as a maintenance downtime, etc.).
So, that was my next question, so a consumer rebalance will trigg
Hey Jason,
Is it an option for you to do the following:
- Bounce in a config change to the brokers to turn off auto-create
- (Batch)-delete the topic(s)
- Wait long enough for consumers to rebalance (after which they will
no longer consume the topic(s))
- Bounce in a config change to the broker
Hi, Jason,
I am not sure that we should delay the 0.8.2 release. The reasons are (1)
There are other features such as the new java producer and Kafka-based
offset management that are potentially useful to people. (2) It may take
some time to completely fix the issue with deleting topic since it in
I think this shortcoming should probably delay release of 0.8.2 until
resolved, no?
On Mon, Jan 26, 2015 at 2:01 PM, Guozhang Wang wrote:
> It then seems to me that delete-topic will not actually work "smoothly"
> until create topic request is added since it is too much to require people
> to tu
So, this is rather disappointing, especially since topic deletion is really
the primary feature in 0.8.2 I'm interested in. The topic I was trying to
delete above had no data for many months. The consumer which is triggering
recreation of that topic has been restarted several times since that top
It then seems to me that delete-topic will not actually work "smoothly"
until create topic request is added since it is too much to require people
to turn off their clients while deleting topics. In this case shall we make
it clear in the release docs of in 0.8.2, or even still mark it as
not-suppo
unsubscribe
On Mon, Jan 26, 2015 at 8:14 AM, Harsha wrote:
> Jun,
> I made an attempt at fixing that issue as part of this JIRA
> https://issues.apache.org/jira/browse/KAFKA-1507 .
> As Jay pointed out there should be admin api if there is more info on
> this api I am interested in a
Jun,
I made an attempt at fixing that issue as part of this JIRA
https://issues.apache.org/jira/browse/KAFKA-1507 .
As Jay pointed out there should be admin api if there is more info on
this api I am interested in adding/fixing this issue.
Thanks,
Harsha
On Mon, Jan 26, 2015, at 07:28
Yes, that's the issue. Currently, topics can be auto-created on
TopicMetadataRequest, which can be issued from both the producer and the
consumer. To prevent that, you would need to stop the producer and the
consumer before deleting a topic. We plan to address this issue once we
have a separate req
There could be another case where if you have auto.create.topics.enable
to set to true ( its true by default) . Any TopicMetadataRequest can
recreate topics. So if you issued a delete topic command and you have
producers running or consumers? too which is issuing a
TopicMetadataRequest than the top
cversion did change (incremented by 2) when I issue the delete command.
>From the logs on the conroller broker (also the leader for the topic), it
looks like the delete proceeds, and then the topic gets recreated
immediately (highlighted in yellow). It appears maybe it’s due to a
consumer client a
Could you do a get on /admin/deleted_topics and see if its cversion changes
after issuing the delete topic command? This will tell us whether the
path /admin/deleted_topics//mytopic
is actually created or not. Anything in the controller log that indicates a
delete topic event is fired?
Thanks,
Ju
yes
On Mon, Jan 26, 2015 at 12:18 AM, Jun Rao wrote:
> Do you have delete.topic.enable turned on in all brokers?
>
> Thanks,
>
> Jun
>
> On Sun, Jan 25, 2015 at 7:56 PM, Jason Rosenberg wrote:
>
> > So far, I have been unable to get delete topic to work, with release
> > candidate 2 for 0.8.2.
Do you have delete.topic.enable turned on in all brokers?
Thanks,
Jun
On Sun, Jan 25, 2015 at 7:56 PM, Jason Rosenberg wrote:
> So far, I have been unable to get delete topic to work, with release
> candidate 2 for 0.8.2.
>
> It worked ok when I ran it in the debugger locally, on a single node
So far, I have been unable to get delete topic to work, with release
candidate 2 for 0.8.2.
It worked ok when I ran it in the debugger locally, on a single node
instance. But when I run it in our staging environment, it is not
successfully even marking the topic for delete, for some reason.
I am
21 matches
Mail list logo
