flatMap() is a transformation only. Calling it by itself does nothing,
and it just describes the relationship between one RDD and another.
You should see it swing into action if you invoke an action, like
count(), on the words RDD.
On Mon, Aug 25, 2014 at 6:32 PM, Steve Lewis <lordjoe2...@gmail.com> wrote:
> I was able to get JavaWordCount running with a local instance under
> IntelliJ.
>
> In order to do so I needed to use maven to package my code and
> call
> String[] jars = {
> "/SparkExamples/target/word-count-examples_2.10-1.0.0.jar" };
> sparkConf.setJars(jars);
>
> After that the sample ran properly and in the debugger I could set break
> points in the main.
>
> However when I do
> something like
> JavaRDD<String> words = lines.flatMap( new WordsMapFunction());
>
> where WordsMapFunction is a separate class like
>
> public static class WordsMapFunction implements FlatMapFunction<String,
> String> {
> private static final Pattern SPACE = Pattern.compile(" ");
> public Iterable<String> call(String s) {
> String[] split = SPACE.split(s);
> for (int i = 0; i < split.length; i++) {
> split[i] = toUpperCase(split[i]);
> }
> return Arrays.asList(split);
> }
> }
>
> Breakpoints set in WordsMapFunction are never hit.
>
> Most interesting functionality in the problems I am trying to solve if in
> the FlatMapFunction and the Function2 code and this is the functionality I
> will need to examine in more detail.
>
> Has anyone figured out how to configure a project to hit breakpoints in
> these functions??
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
