Hi
Is it possible to write custom RDD in java?
Requirement is - I am having a list of Sqlserver tables need to be dumped
in HDFS.
So I have a
List<String> tables = {dbname.tablename,dbname.tablename2......};
then
JavaRDD<String> rdd = javasparkcontext.parllelise(tables);
JavaRDDString> tablecontent = rdd.map(new
Function<String,Iterable<String>>){fetch table and return populate iterable}
tablecontent.storeAsTextFile("hffs path");
In rdd.map(new Function<String,>). I cannot keep complete table content in
memory , so I want to creat my own RDD to handle it.
Thanks
Shushant