Re: apache-spark: Converting List of Rows into Dataset Java

2017-03-30 Thread Karin Valisova
Looks like the parallelization into RDD was the right move I was omitting, JavaRDD jsonRDD = new JavaSparkContext(sparkSession. sparkContext()).parallelize(results); then I created a schema as List fields = new ArrayList(); fields.add(DataTypes.createStructField("column_name1", DataTypes.String

Re: apache-spark: Converting List of Rows into Dataset Java

2017-03-28 Thread Richard Xin
Maybe you could try something like that:        SparkSession sparkSession = SparkSession     .builder()     .appName("Rows2DataSet")     .master("local")     .getOrCreate();         List results = new LinkedList();         JavaRDD jsonRDD =