Hi SK,
In the code you reference,
> status => status.getText.split("
> ").filter(_.startsWith("#")))
gets the text of the tweet and splits it at whitespace. Once the tweet is split
into words, it then filters out all words that don’t start with ‘#’ (e.g., that
aren’t hashtags), and returns those words back to the flatMap transformation.
In your code, you don’t want to do a map, since you’re not changing the tweets.
Instead, you want to do a filter, which will keep all tweets that pass a
predicate function, i.e.,
> stream.map(status =>
> status.getText.filter(_.contains("@Delta")))
should be:
> stream.filter(status =>
> status.getText.contains("@Delta"))
Regards,
Frank Austin Nothaft
[email protected]
[email protected]
202-340-0466
On Oct 9, 2014, at 5:22 PM, SK <[email protected]> wrote:
> Hi,
> I am using Spark 1.1.0. Is there a way to get the complete tweets
> corresponding to a handle (for e.g. @Delta)? I tried using the following
> example that extracts just the hashtags and replaced the "#" with "@" as
> follows. I need the complete tweet and not just the tags.
>
> // val hashTags = stream.flatMap(status => status.getText.split("
> ").filter(_.startsWith("#")))
>
> // replaced the above with the following:
> val tweets = stream.map(status =>
> status.getText.filter(_.contains("@Delta")))
>
> But I get an error: " value contains is not a member of Char"
>
> I am trying to find out if there a better Spark API to get the tweets for
> a handle so that we dont have to do the filtering - Something along the
> lines of searchTwitter(topic, number_of_tags) API that is provided by the
> twitteR package in R would be appropriate.
>
> thanks
>
>
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/getting-tweets-for-a-specified-handle-tp16085.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
