I think your understanding is correct. There will be external libraries that allow you to use the twitter streaming dstream API even in 2.0 though.
On Sat, May 28, 2016 at 8:37 AM, Ricardo Almeida < ricardo.alme...@actnowib.com> wrote: > As far as I could understand... > 1. Using Python (PySpark), the use of Twitter Streaming (TwitterUtils > <http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.streaming.twitter.TwitterUtils$>) > as well as Customer Receivers is restricted to Scala and Java APIs on Spark > 1.6.1; > 2. Maven linking Twitter/spark-streaming-twitter_2.10 is being removed > from Spark Streaming core Scala/Java API (Maven Linking > <http://home.apache.org/~pwendell/spark-releases/spark-2.0.0-preview-docs/streaming-programming-guide.html#linking> > ) > 3. There are no plans to support Twitter Streaming Python API (pyspark) on > Spark 2.0 or later > 4. Twitter Streaming API usage over Python (Spark pyspark) is and will > continue being restricted to Twitter REST API > > > Could you please confirm if these 4 my assumption are correct or is there > any alternative way for using Twitter Streaming API on PySpark from Spark > 2.0? >
