Hi Group
Sorry to rekindle this thread.
Using Spark 1.6.0 on CDH 5.7.
Any idea?
Best
Ayan
On Fri, Oct 7, 2016 at 5:08 PM, Mich Talebzadeh
wrote:
> Hi Ayan,
>
> Depends on the version of Spark you are using.
>
> Have you tried updating stats in Hive?
>
> ANALYZE TABLE ${DATABASE}.${TABLE} PA
Hi Ayan,
Depends on the version of Spark you are using.
Have you tried updating stats in Hive?
ANALYZE TABLE ${DATABASE}.${TABLE} PARTITION (${PARTITION_NAME}) COMPUTE
STATISTICS FOR COLUMNS
and then do
show create table ${TABLE}
HTH
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.co
Posting with correct subject.
On Fri, Oct 7, 2016 at 12:37 PM, ayan guha wrote:
> Hi
>
> Faced one issue:
>
> - Writing Hive Partitioned table using
>
> df.withColumn("partition_date",to_date(df["INTERVAL_DATE"]))
> .write.partitionBy('partition_date').saveAsTable("sometable"
> ,mode="overwr
