Nice
From: Alexander Krasnukhin
Date: Tuesday, March 29, 2016 at 10:42 AM
To: Andrew Davidson
Cc: "user @spark"
Subject: Re: looking for an easy to to find the max value of a column in a
data frame
> You can even use the fact that pyspark has dynamic properties
>
>
col[id]")).collect()
>>
>> max = maxRow[0].asDict()['max(col[id])']
>>
>> max
>>
>> Out[19]:
>>
>> 713912692155621376
>>
>>
>> From: Alexander Krasnukhin
>> Date: Monday, March 28, 2016 at 5:55 PM
>> To: A
])']
>
> max
>
> Out[19]:
>
> 713912692155621376
>
>
> From: Alexander Krasnukhin
> Date: Monday, March 28, 2016 at 5:55 PM
> To: Andrew Davidson
> Cc: "user @spark"
> Subject: Re: looking for an easy to to find the max value of a column in
>
ot;
Subject: Re: looking for an easy to to find the max value of a column in a
data frame
> e.g. select max value for column "foo":
>
> from pyspark.sql.functions import max, col
> df.select(max(col("foo"))).show()
>
> On Tue, Mar 29, 2016 at 2:15
e.g. select max value for column "foo":
from pyspark.sql.functions import max, col
df.select(max(col("foo"))).show()
On Tue, Mar 29, 2016 at 2:15 AM, Andy Davidson <
a...@santacruzintegration.com> wrote:
> I am using pyspark 1.6.1 and python3.
>
>
> *Given:*
>
> idDF2 = idDF.select(idDF.id, idDF
I am using pyspark 1.6.1 and python3.
Given:
idDF2 = idDF.select(idDF.id, idDF.col.id )
idDF2.printSchema()
idDF2.show()
root
|-- id: string (nullable = true)
|-- col[id]: long (nullable = true)
+--+--+
|id| col[id]|
+--+--+
|1008930924| 534494917|
|1
