> Subhash
>
> Sent from my iPhone
>
> > On Mar 7, 2017, at 6:37 AM, El-Hassan Wanas
> wrote:
> >
> > As an example, this is basically what I'm doing:
> >
> > val myDF =
> > originalDataFrame.select(col(columnName).when(col(columnName)
nt to modify the sql statement to
extract the data in the right format and push some preprocessing to the
database.
On 7 Mar 2017, at 12:04, El-Hassan Wanas wrote:
Hello,
There is, as usual, a big table lying on some JDBC data source. I am doing some
data processing on that data from Spark, ho
Hello,
There is, as usual, a big table lying on some JDBC data source. I am
doing some data processing on that data from Spark, however, in order to
speed up my analysis, I use reduced encodings and minimize the general
size of the data before processing.
Spark has been doing a great job at
