I am trying to understand on how spark's sc.textFile() works. I specifically have the question on how it translates the paths with regex in it.
For example:
files = sc.textFile("hdfs://<server>:<port>/file1/*/*/*/*.txt")
How does it find all the sub-directories and recurses to all the leaf
files. ? Is there any documentation on how this happens ?
Pradeep
