Are you looking for this? https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRelation.scala#L69
On Wed, Nov 4, 2015 at 5:11 AM, Tóth Zoltán <t...@looper.hu> wrote: > Hi, > > I'd like to write a parquet file from the driver. I could use the HDFS API > but I am worried that it won't work on a secure cluster. I assume that the > method the executors use to write to HDFS takes care of managing Hadoop > security. However, I can't find the place where HDFS write happens in the > spark source. > > Please help me: > 1.How to write parquet from the driver using the Spark API? > 2. If this wouldn't possible, where can I find the method executors use to > write to HDFS? > > Thanks, > Zoltan > >
