Hi All Has anyone done this with Java API?
On Fri, Aug 5, 2016 at 5:36 PM, Aseem Bansal <asmbans...@gmail.com> wrote: > I need to use few columns out of a csv. But as there is no option to read > few columns out of csv so > 1. I am reading the whole CSV using SparkSession.csv() > 2. selecting few of the columns using DataFrame.select() > 3. applying schema using the .as() function of Dataset<Row>. I tried to > extent org.apache.spark.sql.Encoder as the input for as function > > But I am getting the following exception > > Exception in thread "main" java.lang.RuntimeException: Only expression > encoders are supported today > > So my questions are - > 1. Is it possible to read few columns instead of whole CSV? I cannot > change the CSV as that is upstream data > 2. How do I apply schema to few columns if I cannot write my encoder? >
