Re: [PySpark SQL] New column with the maximum of multiple terms?

Sorry, I didn't try that. On Fri, Feb 24, 2023 at 4:13 PM Russell Jurney wrote: > Oliver, just curious: did you get a clean error message when you broke it > out into separate statements? > > Thanks, > Russell Jurney @rjurney > russell.jur...@gmail.com LI

Re: [PySpark SQL] New column with the maximum of multiple terms?

Oliver, just curious: did you get a clean error message when you broke it out into separate statements? Thanks, Russell Jurney @rjurney russell.jur...@gmail.com LI FB datasyndrome.com Book a time on Ca

Re: SPIP architecture diagrams

Sounds like I have to decide for myself what to use. A correction Vision should read* Visio * ideally the SPIP guide https://spark.apache.org/improvement-proposals.html should include this topic. Additionally there should be a repository for the original diagrams as well. From the said guide:

Re: [PySpark SQL] New column with the maximum of multiple terms?

Hello, Thanks for the advice. First of all, it looks like I used the wrong *max* function, but *pyspark.sql.functions.max* isn't right either, because it finds the maximum of a given column over groups of rows. To find the maximum among multiple columns, I need *pyspark.sql.functions.greate

Re: Unable to handle bignumeric datatype in spark/pyspark

Hi Nidhi, can you create a BigQuery table with a bignumeric and numeric column types, add a few lines and try to read into spark. through DF and do df.printSchema() df.show(5,False) HTH view my Linkedin profile https://en.ev