There is this thread on Stack Overflow <http://stackoverflow.com/questions/23995040/write-to-multiple-outputs-by-key-spark-one-spark-job> about the same topic, which you may find helpful.
On Wed, Oct 1, 2014 at 11:17 AM, Nicholas Chammas < [email protected]> wrote: > Not that I'm aware of. I'm looking for a work-around myself! > > On Wed, Oct 1, 2014 at 11:15 AM, Tomer Benyamini <[email protected]> > wrote: > >> Yes exactly.. so I guess this is still an open request. Any workaround? >> >> On Wed, Oct 1, 2014 at 6:04 PM, Nicholas Chammas >> <[email protected]> wrote: >> > Are you trying to do something along the lines of what's described here? >> > https://issues.apache.org/jira/browse/SPARK-3533 >> > >> > On Wed, Oct 1, 2014 at 10:53 AM, Tomer Benyamini <[email protected]> >> > wrote: >> >> >> >> Hi, >> >> >> >> I'm trying to write my JavaPairRDD using saveAsNewAPIHadoopFile with >> >> MultipleTextOutputFormat,: >> >> >> >> outRdd.saveAsNewAPIHadoopFile("/tmp", String.class, String.class, >> >> MultipleTextOutputFormat.class); >> >> >> >> but I'm getting this compilation error: >> >> >> >> Bound mismatch: The generic method saveAsNewAPIHadoopFile(String, >> >> Class<?>, Class<?>, Class<F>) of type JavaPairRDD<K,V> is not >> >> applicable for the arguments (String, Class<String>, Class<String>, >> >> Class<MultipleTextOutputFormat>). The inferred type >> >> MultipleTextOutputFormat is not a valid substitute for the bounded >> >> parameter <F extends OutputFormat<?,?>> >> >> >> >> I bumped into some discussions suggesting to use MultipleOutputs >> >> >> >> ( >> http://hadoop.apache.org/docs/r2.3.0/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html >> ), >> >> but this also fails from the same reason. >> >> >> >> Would love some assistance :) >> >> >> >> Thanks, >> >> Tomer >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: [email protected] >> >> For additional commands, e-mail: [email protected] >> >> >> > >> > >
