Re: Save hive table from spark in hive 2.1.0

2017-12-10 Thread Alejandro Reina
I did what you said and I was finally able to update the scheme. But you're right, it's very dirty, I have to modify almost all the scripts. The problem of the scripts comes from having already a previous table in that version, many of the tables or columns that I try to add, already exist and it g

Re: Save hive table from spark in hive 2.1.0

2017-12-10 Thread रविशंकर नायर
Hi, Good try. As you can see, when you run upgrade using schematool, there is a duplicate column error. Can you please look the script generated and edit to avoid duplicate column? Not sure why the Hive guys made it complicated, I did face same issues like you. Can anyone else give a clean and b

Re: Save hive table from spark in hive 2.1.0

2017-12-10 Thread Alejandro Reina
I have tried what you propose, added the property to hive-site.xml, and although with this option I can run hive, this does not solve my problem. I'm sorry if maybe you explain me wrongly. I need to save a dataframe transformed into spark in hive, with the version of scheme 2.1.1 of hive (last sta