Note that Spark never guarantees ordering of columns. There’s nothing in Spark
documentation that says that the columns will be ordered a certain way. The
proposed solution relies on an implementation detail that might change in
future version of Spark.
Ideally, you shouldn’t rely on Dataframe
Ohh
Thanks a lot
On Thu, Nov 12, 2020, 21:23 Subash Prabakar
wrote:
> Hi Vikas,
>
> He suggested to use the select() function after your withColumn function.
>
> val ds1 = ds.select("Col1", "Col3").withColumn("Col2",
> lit("sample”)).select(“Col1”, “Col2”, “Col3")
>
>
> Thanks,
> Subash
>
>
Hi Vikas,
He suggested to use the select() function after your withColumn function.
val ds1 = ds.select("Col1", "Col3").withColumn("Col2",
lit("sample”)).select(“Col1”, “Col2”, “Col3")
Thanks,
Subash
On Thu, Nov 12, 2020 at 9:19 PM Vikas Garg wrote:
> I am deriving the col2 using with colunn
You can still simply select the columns by name in order, after
.withColumn()
On Thu, Nov 12, 2020 at 9:49 AM Vikas Garg wrote:
> I am deriving the col2 using with colunn which is why I cant use it like
> you told me
>
> On Thu, Nov 12, 2020, 20:11 German Schiavon
> wrote:
>
>> ds.select("Col1"
I am deriving the col2 using with colunn which is why I cant use it like
you told me
On Thu, Nov 12, 2020, 20:11 German Schiavon
wrote:
> ds.select("Col1", "Col2", "Col3")
>
> On Thu, 12 Nov 2020 at 15:28, Vikas Garg wrote:
>
>> In Spark Datase, if we add additional column using
>> withColumn
>
ds.select("Col1", "Col2", "Col3")
On Thu, 12 Nov 2020 at 15:28, Vikas Garg wrote:
> In Spark Datase, if we add additional column using
> withColumn
> then the column is added in the last.
>
> e.g.
> val ds1 = ds.select("Col1", "Col3").withColumn("Col2", lit("sample"))
>
> the the order of column
