Hi, Cody's right.
subscribe - Topic subscription strategy that accepts topic names as a comma-separated string, e.g. topic1,topic2,topic3 [1] subscribepattern - Topic subscription strategy that uses Java’s java.util.regex.Pattern for the topic subscription regex pattern of topics to subscribe to, e.g. topic\d [2] [1] https://jaceklaskowski.gitbooks.io/spark-structured-streaming/spark-sql-streaming-KafkaSource.html#subscribe [2] https://jaceklaskowski.gitbooks.io/spark-structured-streaming/spark-sql-streaming-KafkaSource.html#subscribepattern Pozdrawiam, Jacek Laskowski ---- https://about.me/JacekLaskowski Spark Structured Streaming (Apache Spark 2.2+) https://bit.ly/spark-structured-streaming Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Tue, Sep 19, 2017 at 10:34 PM, Cody Koeninger <c...@koeninger.org> wrote: > You should be able to pass a comma separated string of topics to > subscribe. subscribePattern isn't necessary > > > > On Tue, Sep 19, 2017 at 2:54 PM, kant kodali <kanth...@gmail.com> wrote: > > got it! Sorry. > > > > On Tue, Sep 19, 2017 at 12:52 PM, Jacek Laskowski <ja...@japila.pl> > wrote: > >> > >> Hi, > >> > >> Use subscribepattern > >> > >> You haven't googled well enough --> > >> https://jaceklaskowski.gitbooks.io/spark-structured- > streaming/spark-sql-streaming-KafkaSource.html > >> :) > >> > >> Pozdrawiam, > >> Jacek Laskowski > >> ---- > >> https://about.me/JacekLaskowski > >> Spark Structured Streaming (Apache Spark 2.2+) > >> https://bit.ly/spark-structured-streaming > >> Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark > >> Follow me at https://twitter.com/jaceklaskowski > >> > >> On Tue, Sep 19, 2017 at 9:50 PM, kant kodali <kanth...@gmail.com> > wrote: > >>> > >>> HI All, > >>> > >>> I am wondering How to read from multiple kafka topics using structured > >>> streaming (code below)? I googled prior to asking this question and I > see > >>> responses related to Dstreams but not structured streams. Is it > possible to > >>> read multiple topics using the same spark structured stream? > >>> > >>> sparkSession.readStream() > >>> .format("kafka") > >>> .option("kafka.bootstrap.servers", "localhost:9092") > >>> .option("subscribe", "hello1") > >>> .option("startingOffsets", "earliest") > >>> .option("failOnDataLoss", "false") > >>> .load(); > >>> > >>> > >>> Thanks! > >> > >> > > >
