Re: can we all help use our expertise to create an IT solution for Covid-19

2020-03-26 Thread Laurent Bastien Corbeil
People in tech should be more humble and admit this is not something they can fix. There's already plenty of visualizations, dashboards etc showing the spread of the virus. This is not even a big data problem, so Spark would have limited use. On Thu, Mar 26, 2020 at 10:37 AM Sol Rodriguez wrote:

Temporary tables for Spark SQL

2019-11-12 Thread Laurent Bastien Corbeil
Hello, I am new to Spark, so I have a basic question which I couldn't find an answer online. If I want to run SQL queries on a Spark dataframe, do I have to create a temporary table first? I know I could use the Spark SQL API, but is there a way of simply reading the data and run SQL queries wit