we are on 1.6.1 version of spark under CDH5.7.1
On Mon, Feb 6, 2017 at 2:53 PM, Xiao Li wrote:
> Which Spark version are you using?
>
> 2017-02-06 12:25 GMT-05:00 vaquar khan :
>
>> Did you try MSCK REPAIR TABLE ?
>>
>> Regards,
>> Vaquar Khan
>>
>> On Feb 6, 2017 11:21 AM, "KhajaAsmath Mohamm
Which Spark version are you using?
2017-02-06 12:25 GMT-05:00 vaquar khan :
> Did you try MSCK REPAIR TABLE ?
>
> Regards,
> Vaquar Khan
>
> On Feb 6, 2017 11:21 AM, "KhajaAsmath Mohammed"
> wrote:
>
>> I dont think so, i was able to insert overwrite other created tables in
>> hive using spark
Tried below in spark shell and in dataframes. None of them worked. Can
access same view in HUE.
scala> hiveObj.refreshTable("dtmlab.vehscan_jackwagon_xml_mart_view")
scala> val sample = sqlContext.sql("select * from
dtmlab.vehscan_jackwagon_xml_mart_view").collect()
org.apache.spark.sql.Analysis
Did you try MSCK REPAIR TABLE ?
Regards,
Vaquar Khan
On Feb 6, 2017 11:21 AM, "KhajaAsmath Mohammed"
wrote:
> I dont think so, i was able to insert overwrite other created tables in
> hive using spark sql. The only problem I am facing is, spark is not able
> to recognize hive view name. Very
I dont think so, i was able to insert overwrite other created tables in
hive using spark sql. The only problem I am facing is, spark is not able
to recognize hive view name. Very strange but not sure where I am doing
wrong in this.
On Mon, Feb 6, 2017 at 11:03 AM, Jon Gregg wrote:
> Confirming
Confirming that Spark can read newly created views - I just created a test
view in HDFS and I was able to query it in Spark 1.5 immediately after
without a refresh. Possibly an issue with your Spark-Hive connection?
Jon
On Sun, Feb 5, 2017 at 9:31 PM, KhajaAsmath Mohammed <
mdkhajaasm...@gmail.c
Hi Khan,
It didn't work in my case. used below code. View is already present in Hive
but I cant read that in spark sql. Throwing exception that table not found
sqlCtx.refreshTable("schema.hive_view")
Thanks,
Asmath
On Sun, Feb 5, 2017 at 7:56 PM, vaquar khan wrote:
> Hi Ashmath,
>
> Try r
Hi Ashmath,
Try refresh table
// spark is an existing SparkSession
spark.catalog.refreshTable("my_table")
http://spark.apache.org/docs/latest/sql-programming-guide.html#metadata-refreshing
Regards,
Vaquar khan
On Sun, Feb 5, 2017 at 7:19 PM, KhajaAsmath Mohammed <
mdkhajaasm...@gmail.com
Hi,
I have a hive view which is basically set of select statements on some
tables. I want to read the hive view and use hive builtin functions
available in spark sql.
I am not able to read that hive view in spark sql but can retreive data in
hive shell.
can't spark access hive views?
Thanks,
As
