Yes, this can be done. quick python equivalent:
# In Driver
fileList=["/file1.txt","/file2.txt"]
rdd = []
for f in fileList:
rdd = jsc.textFile(f)
rdds.append(rdd)
On Sat, Oct 31, 2015 at 11:09 PM, amit tewari <[email protected]>
wrote:
> Hi
>
> I need the ability to be able to create RDDs programatically inside my
> program (e.g. based on varaible number of input files).
>
> Can this be done?
>
> I need this as I want to run the following statement inside an iteration:
>
> JavaRDD<String> rdd1 = jsc.textFile("/file1.txt");
>
> Thanks
> Amit
>
--
Best Regards,
Ayan Guha
