Hi Sean,
I'd just like to take the first "word" of every line, and use it as a
variable for later. Is there a way to do that?
Here's the gist of what I want to do:
val lines = KafkaUtils.createStream(ssc, "localhost:2181", "test",
Map("test" -> 10)).map(_._2)
val words = lines.flatMap(_.split(" "))
val acct = words.filter(word => word.startsWith("SECRETWORD"))
val pairs = words.map(word => (acct+"_"+word, 1))
Take all lines coming into Kafka, and add the word 'acct' to each word.
As an example, here is a line:
"hello world you are SECRETWORDthebest hello world"
And it should do this:
(SECRETWORDthebest_hello, 2), (SECRETWORDthebest_world, 2),
(SECRETWORDthebest_you, 1), etc...
Harold
On Wed, Oct 29, 2014 at 3:36 PM, Sean Owen <[email protected]> wrote:
> What would it mean to make a DStream into a String? it's inherently a
> sequence of things over time, each of which might be a string but
> which are usually RDDs of things.
>
> On Wed, Oct 29, 2014 at 11:15 PM, Harold Nguyen <[email protected]>
> wrote:
> > Hi all,
> >
> > How do I convert a DStream to a string ?
> >
> > For instance, I want to be able to:
> >
> > val myword = words.filter(word => word.startsWith("blah"))
> >
> > And use "myword" in other places, like tacking it onto (key, value)
> pairs,
> > like so:
> >
> > val pairs = words.map(word => (myword+"_"+word, 1))
> >
> > Thanks for any help,
> >
> > Harold
> >
> >
> >
> >
>
