Easy way of doing it
newdf = df.withColumn('total', sum(df[col] for col in df.columns))
On 22 April 2016 at 11:51, Naveen Kumar Pokala
wrote:
> Hi,
>
>
>
> Do we have any way to perform Row level operations in spark dataframes.
>
>
>
>
>
> For example,
>
>
>
> I have a dataframe with columns f
You can define your own udf, following is one example
Thanks
Zhan Zhang
val foo = udf((a: Int, b: String) => a.toString + b)
checkAnswer(
// SELECT *, foo(key, value) FROM testData
testData.select($"*", foo('key, 'value)).limit(3),
On Apr 21, 2016, at 8:51 PM, Naveen Kumar Pokala
mailt
