That will work. We have some utility classes for exposing the ZK-info. On Fri, Sep 18, 2015 at 10:50 AM, Robert Metzger <rmetz...@apache.org> wrote:
> Hi Jakob, > > currently, its not possible to subscribe to multiple topics with one > FlinkKafkaConsumer. > > So for now, you have to create a FKC for each topic .. so you'll end up > with 50 sources. > > As soon as Kafka releases the new consumer, it will support subscribing to > multiple topics (I think even with pattern support) and we can easily > expose the APIs also to the FlinkKafkaConsumer. > As you can see here: > https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan > Kafka has plans to release the new consumer API in October. > As soon as the new API is out, we'll support it. > > I hope this solution is okay for you. If not, please let me know ;) > > > Robert > > On Fri, Sep 18, 2015 at 10:43 AM, Jakob Ericsson <jakob.erics...@gmail.com > > wrote: > >> Hi, >> >> Would it be possible to get the FlinkKafkaConsumer to support multiple >> topics, like a list? >> >> Or would it be better to instantiate one FlinkKafkaConsumers per topic >> and add as a source? >> We have about 40-50 topics to listen for one job. >> Or even better, supply a regexp pattern that defines the queues, this >> means that you have to do some queries against ZK to get information about >> topics. >> >> Kind regards, >> Jakob >> > >
