The most convenient way I'm aware of from Java is to use createDataset and pass Encoder.String
That gives you a Dataset<String> if you still want Dataset<Row> the you can call .toDF on it On Thu, 18 Nov 2021, 10:27 , <mar...@wunderlich.com> wrote: > Hello, > > I am struggling with a task that should be super simple: I would like to > create a Spark DF of Type Dataset<Row> with one column from a single String > (or from a one-element List of Strings). The column header should be "text". > > SparkContext.parallelize() does not work, because it returns RDD<T> and > not Dataset<Row> and it takes a "ClassTag" as 3rd parameter. > > I am able to convert a List of Strings to JavaRDD<Row> using this: > JavaSparkContext javaSparkContext = new JavaSparkContext(sparkContext); > > JavaRDD<String> javaRdd = javaSparkContext.parallelize(textList); > > But then I am stuck with this javaRDD. Besides, it seems overly complex > having to create an intermediate representation. > > There is also this SO post with a solution in Scala that I have not been > able to convert to Java, because the APIs differ: > > > https://stackoverflow.com/questions/44028677/how-to-create-a-dataframe-from-a-string > > Basically, what I am looking for is something simple like: > > Dataset<Row> myData = sparkSession.createDataFrame(textList, "text"); > > Any hints? Thanks a lot. > > Cheers, > > Martin >
