thanks Jorn.
Is the data stored in hdfs directory in binary format and can spark use it
or needs to convert into json etc. I am not familiar with the nature of the
twitter logs.
in short what tool I can use to convert the log files into useful format
and format would that be?
thanks
Dr Mich Tal
Or combine both! It is possible with Spark Streaming to combine streaming data
and on HDFS. In the end it always depends what you want to do and when you need
what.
> On 03 Jun 2016, at 10:26, Mich Talebzadeh wrote:
>
> I use twitter data with spark streaming to experiment with twitter data.
I use twitter data with spark streaming to experiment with twitter data.
Basic stuff
val ssc = new StreamingContext(sparkConf, Seconds(2))
val tweets = TwitterUtils.createStream(ssc, None)
val statuses = tweets.map(status => status.getText())
statuses.print()
Another alternative
