Jayesh, Thanks, but AFAICT that's just the metadata "shell" -- the Cluster object just holds information about a cluster, it has no functionality to actually retrieve it. I looked briefly around the source and couldn't find a (public) API that might fill out the Cluster object. KafkaConsumer has stuff like:
Cluster cluster = this.metadata.fetch();
But that's all private...
On Tue, Oct 25, 2016 at 02:11:14PM +0000, Thakrar, Jayesh wrote:
> Have a look at the Cluster which has a "topic" method to get a set of all the
> topics.
>
> https://protect-us.mimecast.com/s/LLEEBwuodaEbCd
>
> In version 8/9, there was also the ZKUtils, but the desire is to have clients
> not to interrogate ZK directly.
>
> On 10/24/16, 4:32 PM, "Ben Osheroff" <b...@zendesk.com.INVALID> wrote:
>
> Hiya!
>
> I've been trying to merge
> https://protect-us.mimecast.com/s/GWddBYHl3K5DFR,
> which adds a much-requested feature of Maxwell, that of being able to
> have a topic-per-mysql-table. When we receive a row we programmatically
> generate the topic name, and the first thing we do is call
> `KafkaProducer#partitionsFor(topic)`, so that we know how to partition
> the data.
>
> The problem I'm running into is in trying to detect the case where a
> topic doesn't exist. If auto-creation is on, `partitionsFor()` seems to
> correctly auto-create the topic, but if auto-creation is off the
> behavior is kinda wonky; kafka goes into a metadata-fetch loop, logging
>
> "Error while fetching metadata with correlation id 573
> :{topic=UNKNOWN_TOPIC_OR_PARTITION}"
>
> but then ultimately throwing me back a `TimeoutException` after 60 tries
> or so.
>
> I can rescue/rethrow the TimeoutException, but it seems like there might
> be a better way that I'm missing. Any ideas? I'd ideally just like a
> way to fail fast and clean when the topic doesn't exist (and
> auto-creation is off).
>
> Thanks,
> Ben Osheroff
> zendesk.com
>
>
>
>
>
signature.asc
Description: PGP signature
