Re: pyspark.sql.functions ide friendly

2019-04-16 Thread 880f0464
Hi. That's a problem with Spark as such and in general can be addressed on IDE to IDE basis - see for example https://stackoverflow.com/q/40163106 for some hints. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Tuesday, April 16, 2019 2:10 PM, educhana wrote: > Hi, > >

UnresolvedException: Invalid call to dataType on unresolved object

2018-05-01 Thread 880f0464
Hi Everyone, I wonder If someone could be so kind and share some light on this problem: [UnresolvedException: Invalid call to dataType on unresolved object when using DataSet constructed from Seq.empty (since Spark 2.3.0)](https://stackoverflow.com/q/49757487) Cheers, A. Sent with [ProtonMail

spark.python.worker.reuse not working as expected

2018-05-01 Thread 880f0464
Hi Everyone, I wonder If someone could be so kind and share some light on this problem: [spark.python.worker.reuse not working as expected](https://stackoverflow.com/q/50043684) Cheers, A. Sent with [ProtonMail](https://protonmail.com) Secure Email.

PySpark.sql.filter not performing as it should

2018-05-01 Thread 880f0464
Hi Everyone, I wonder If someone could be so kind and share some light on this problem: [PySpark.sql.filter not performing as it should](https://stackoverflow.com/q/49995538) Cheers, A. Sent with [ProtonMail](https://protonmail.com) Secure Email.

Re: Naming policy for packages

2018-08-15 Thread 880f0464
IANAL, however in case of any legal action ASAF might have a pretty weak case for at least two reasons: - Spark is a common word and its usage in the names of software projects (in different) form is widespread and precedes release of Apache Spark. For example outside data processing community it'