The destination table is partitioned. If I don’t specify the columns I get :
Error: org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException: Partition column name year
conflicts with table columns. (state=,code=0)
younes
From: Tejas Patil [mailto:tejas.patil...@gmail.com]
Sent: January-26-16 11:39 AM
To: Younes Naguib
Cc: user@spark.apache.org
Subject: Re: ctas fails with "No plan for CreateTableAsSelect"
In CTAS, you should not specify the column information as it is derived from
the result of SELECT statement. See
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTableAsSelect(CTAS)
~tejasp
On Tue, Jan 26, 2016 at 9:48 PM, Younes Naguib
<younes.nag...@tritondigital.com<mailto:younes.nag...@tritondigital.com>> wrote:
The CTAS works when not using partitions or not defining columns.
Ex: Create table default.tab1 stored as parquet location
'hdfs://mtl2-alabs-dwh01.streamtheworld.net:9000/younes/geo_location_enrichment<http://mtl2-alabs-dwh01.streamtheworld.net:9000/younes/geo_location_enrichment>'
as
Select *
from tab2
Works.
From: Ted Yu [mailto:yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>]
Sent: January-26-16 11:11 AM
To: Younes Naguib
Cc: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: ctas fails with "No plan for CreateTableAsSelect"
Maybe try enabling the following (false by default):
"spark.sql.hive.convertCTAS"
doc = "When true, a table created by a Hive CTAS statement (no USING clause)
will be " + "converted to a data source table, using the data source set by
spark.sql.sources.default.")
FYI
On Tue, Jan 26, 2016 at 8:06 AM, Younes Naguib
<younes.nag...@tritondigital.com<mailto:younes.nag...@tritondigital.com>> wrote:
SQL on beeline and connecting to the thriftserver.
Younes
From: Ted Yu [mailto:yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>]
Sent: January-26-16 11:05 AM
To: Younes Naguib
Cc: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: ctas fails with "No plan for CreateTableAsSelect"
Were you using HiveContext or SQLContext ?
Can you show the complete stack trace ?
Thanks
On Tue, Jan 26, 2016 at 8:00 AM, Younes Naguib
<younes.nag...@tritondigital.com<mailto:younes.nag...@tritondigital.com>> wrote:
Hi,
I’m running CTAS, and it fails with “Error: java.lang.AssertionError: assertion
failed: No plan for CreateTableAsSelect HiveTable....”
Here what my sql looks like :
Create tbl (
Col1 timestamp ,
Col2 string,
Col3 int,
.....
)
partitioned by (year int, month int, day int)
stored as parquet
location 'hdfs://...
as select.....
;
The select by itself works.
I’m running spark 1.6.
Younes
