Hi Team,
I wanted to understand how Hive on Spark actually maps to Spark jobs
underneath triggered by Hive.
AFAIK each Hive query would trigger a new Spark job. But this was
contradicted by someone and wanted to confirm what is the real design
implementation.
Please let me know if there is refere
HI Team,
When I add a column to my data frame using withColumn and assign some
value, it automatically creates the schema with this column to be not
nullable.
My final Hive table schema where I want to insert it has this column to be
nullable and hence throws an error when I try to save.
Is there
Hi Team,
Is there a standard way of generating a unique id for each row in from
Spark SQL. I am looking for functionality similar to UUID generation in
hive.
Let me know if you need any additional information.
Thanks,
Ninad
Hi Team,
When going through Dataset class for Spark 2.0 it comes across that both
overloaded map functions with encoder and without are marked as
experimental.
Is there a reason and issues that developers whould be aware of when using
this for production applications. Also is there a "non-experim
;
> On Tue, Dec 13, 2016 at 10:43 AM, Ninad Shringarpure
> wrote:
>
>
> Hi Team,
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Does Spark 2.0 support non-primitive types in collect_list for inserting
> nested collections?
> Would appreciate any references or samples.
>
> Thanks,
> Ninad
>
>
>
>
>
>
>
>
>
>
Hi Team,
Does Spark 2.0 support non-primitive types in collect_list for inserting
nested collections?
Would appreciate any references or samples.
Thanks,
Ninad
Unsubscribe
Hi Team,
One of my client teams is trying to see if they can use Spark to source
data from RDBMS instead of Sqoop. Data would be substantially large in the
order of billions of records.
I am not sure reading the documentations whether jdbcRDD by design is going
to be able to scale well for this
