Re: Rust Spark Connect

Hi, There isn’t an official client (the only official ones are python/java/scala) but this uses the spark connect gRPC API and is well supported: https://github.com/sjrusso8/spark-connect-rs Ed Elliott From: Tarkhan Shakhbazyan Sent: Sunday, September 29

Re: How can I use pyspark to upsert one row without replacing entire table

You’ll need to do an insert and use a trigger on the table to change it into an upsert, also make sure your mode is append rather than overwrite. Ed From: Siavash Namvar Sent: Wednesday, August 12, 2020 4:09:07 PM To: Sean Owen Cc: User Subject: Re: How can I

Re: Mock spark reads and writes

both approaches and I would favour the second approach of allowing reads and writes to happen on a local spark instance as it tells you so much more than just whether functions were called in the right order and with the right parameters. Ed > On 14 Jul 2020, at 18:18, Dark Crusader wr

Initial job has not accepted any resources (but workers are in UI)

do I tweak to know for sure? Thanks for any help, -Ed ps. most of the searches for this error end up being about workers now connecting but our instance's UI shows the workers. 46692 [pool-10-thread-1] INFO org.apache.hadoop.mapred.FileInputFormat - Total input paths to process : 1 46709 [pool-1