Using flatMap on Dataframes with Spark 2.0

[Julien Nauroy]

Hi, 

I'm trying to call flatMap on a Dataframe with Spark 2.0 (rc5). 
The code is the following: 


var data = spark.read.parquet(fileName).flatMap(x => List(x)) 


Of course it's an overly simplified example, but the result is the same. 
The dataframe schema goes from this: 


root 
|-- field1: double (nullable = true) 
|-- field2: integer (nullable = true) 
(etc) 




to this: 

root 
|-- value: binary (nullable = true) 

Plus I have to provide an encoder for Row. 
I expect to get the same schema after calling flatMap. 
Any idea what I could be doing wrong? 


Best regards, 
Julien