Hi ,
I see ,Does that means Spark structured streaming doesn't work with Twitter
streams ?
I could see people used kafka or other streaming tools and used spark to
process the data in structured streaming .
The below doesn't work directly with Twitter Stream until I set up Kafka ?
> import org.apache.spark.sql.SparkSession
> val spark = SparkSession
> .builder()
> .appName("Spark SQL basic example")
> .config("spark.some.config.option", "some-value")
> .getOrCreate()
> // For implicit conversions like converting RDDs to DataFrames
> import spark.implicits
>>
>> / Read text from socket
>
> val socketDF = spark
>
> .readStream
>
> .format("socket")
>
> .option("host", "localhost")
>
> .option("port", 9999)
>
> .load()
>
>
>> socketDF.isStreaming // Returns True for DataFrames that have
>> streaming sources
>
>
>> socketDF.printSchema
>
>
>
Thanks,
Divya
On 1 February 2018 at 10:30, Tathagata Das <tathagata.das1...@gmail.com>
wrote:
> Hello Divya,
>
> To add further clarification, the Apache Bahir does not have any
> Structured Streaming support for Twitter. It only has support for Twitter +
> DStreams.
>
> TD
>
>
>
> On Wed, Jan 31, 2018 at 2:44 AM, vermanurag <anurag.ve...@fnmathlogic.com>
> wrote:
>
>> Twitter functionality is not part of Core Spark. We have successfully used
>> the following packages from maven central in past
>>
>> org.apache.bahir:spark-streaming-twitter_2.11:2.2.0
>>
>> Earlier there used to be a twitter package under spark, but I find that it
>> has not been updated beyond Spark 1.6
>> org.apache.spark:spark-streaming-twitter_2.11:1.6.0
>>
>> Anurag
>> www.fnmathlogic.com
>>
>>
>>
>>
>> --
>> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>>
>>
>
