Re: access hdfs file name in map()

2014-08-04 Thread Roberto Torella
map_input_file"). > > I was hoping to do something like > sc.textFile("hdfs_path").pipe("""/usr/bin/awk > "{print\"${mapreduce_map_input_file}\",$0}" """). This gives me some weird > kyro buffer overflow exception... Haven'

Re: access hdfs file name in map()

2014-08-01 Thread Roberto Torella
Hi Simon, I'm trying to do the same but I'm quite lost. How did you do that? (Too direct? :) Thanks and ciao, r- -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/access-hdfs-file-name-in-map-tp6551p11160.html Sent from the Apache Spark User List mailing l