By default, I have these:
hadoop.proxyuser.hive.hosts=*
hadoop.proxyuser.hive.groups=*
Also, I have impersonation enabled. These settings should have made the
compaction to run without issues, but does not. What am I missing?
On Wed, Nov 16, 2016 at 6:00 PM, Manoj Murumkar
wrote:
> Okay, made
Okay, made the cleaner work as well :)
When the files are owned by a user that is different from the user that
runs metastore ("hive"), this doesn't work. Just to test this, I changed
ownership of the files for the table to "hive" and ran a major compaction
and it does the job.
drwxrwxrwt - hiv
Further update:
I see this:
+---+--+++-+---++--+
| dbname | tabname | partname | type |state|
workerid | starttime|
+---+--+++-+--
Quick update:
After each compaction, files under base directory (for the buckets) have
latest data. However, I am expecting to see all delta files (and
directories) gone, as they should be merged in the base directory.
Otherwise, we'll start seeing too many small files on HDFS which is a
problem.
Hi,
We are trying to implement transaction feature in hive. I created following
table:
++--+
|
createtab_stmt |
+
Are you loading new data (inserts) or mutating existing data
(update/delete) or both? And by 'transactions' are you referring to Hive
ACID transactional tables? If so:
For new data, I think you should be able to use:
INSERT INTO transactional_table ... FROM table_over_file_to_be_loaded
For upda
Hi,
Is it possible to do bulk load using files in hive table backed by
transactions instead of update statements.
Thanks
ot;user@hive.apache.org"
>>> Date: Friday, October 9, 2015 at 4:05 AM
>>> To: "user@hive.apache.org"
>>> Subject: Storm HiveBolt missing records due to batching of Hive
>>> transactions
>>>
>>>
>
gt; From: Harshit Raikar
>> Reply-To: "user@hive.apache.org"
>> Date: Friday, October 9, 2015 at 4:05 AM
>> To: "user@hive.apache.org"
>> Subject: Storm HiveBolt missing records due to batching of Hive
>> transactions
>>
>>
>&
ush to the HiveBolt using tick tuples that
> would address this situation.
>
> From: Harshit Raikar
> Reply-To: "user@hive.apache.org"
> Date: Friday, October 9, 2015 at 4:05 AM
> To: "user@hive.apache.org"
> Subject: Storm HiveBolt missing records due
ctober 9, 2015 at 4:05 AM
To: "user@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>
Subject: Storm HiveBolt missing records due to batching of Hive transactions
To store the processed records I am using HiveBolt in Storm topology with
followi
To store the processed records I am using HiveBolt in Storm topology with
following arguments.
- id: "MyHiveOptions"
className: "org.apache.storm.hive.common.HiveOptions"
- "${metastore.uri}" # metaStoreURI
- "${hive.database}" # database
ore neither Peridale Ltd, its subsidiaries nor their employees accept any
responsibility.
From: Steve Howard [mailto:stevedhow...@gmail.com]
Sent: 24 September 2015 14:53
To: user@hive.apache.org
Subject: Re: ORA-8177 with Hive transactions
All,
We continue to struggle with th
e/HIVE-11833 that try to
>> address this.
>> We can do a patch similar to the first one; can you file a JIRA?
>>
>> From: Steve Howard
>> Reply-To: "user@hive.apache.org"
>> Date: Friday, September 18, 2015 at 10:54
>> To: "user@hive.apache.org&q
To: user@hive.apache.org
Subject: ORA-8177 with Hive transactions
While troubleshooting an issue with transactions shortly after enabling them, I
noticed the following in an Oracle trace, which is our metastore for hive...
ORA-8177: can't serialize access for this transaction
These w
issues.apache.org/jira/browse/HIVE-11833 that try to address this.
> We can do a patch similar to the first one; can you file a JIRA?
>
> From: Steve Howard
> Reply-To: "user@hive.apache.org"
> Date: Friday, September 18, 2015 at 10:54
> To: "user@hive.apach
r@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>
Date: Friday, September 18, 2015 at 10:54
To: "user@hive.apache.org<mailto:user@hive.apache.org>"
mailto:user@hive.apache.org>>
Subject: ORA-8177 with Hive transactions
While
While troubleshooting an issue with transactions shortly after enabling
them, I noticed the following in an Oracle trace, which is our metastore
for hive...
ORA-8177: can't serialize access for this transaction
These were thrown on "insert into HIVE_LOCKS..."
Traditionally in Oracle, if an appli
ar 5, 2015 at 4:51 PM, David Simoes wrote:
> Ive had some troubles enabling transactions in Hive 1.0.0 and Ive made a
> post in
> http://stackoverflow.com/questions/28867368/hive-transactions-are-crashing
>
> Could anyone check it out and give me some pointers on why things are
&
Ive had some troubles enabling transactions in Hive 1.0.0 and Ive made a post in
http://stackoverflow.com/questions/28867368/hive-transactions-are-crashing
Could anyone check it out and give me some pointers on why things are crashing?
Tyvm, Dave
Thanks,
Yogendra
From: Alan Gates <mailto:ga...@hortonworks.com>
Sent: 12/3/2014 9:51 AM
To: user@hive.apache.org <mailto:user@hive.apache.org>
Subject: Re: hive transactions
Usually this means the metastore database was not properly up
on? If
>>> you upgraded did you run the upgrade scripts? In the Hive 0.12 to Hive
>>> 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql, which will
>>> create these tables. If you're upgrading from 13 to 14 there shouldn't be
>>> a need to c
from 0.13
>> to 0.14.
>>
>> any idea as to wht else might cause this issue?
>>
>> Thanks,
>> Yogendra
>> ------
>> From: Alan Gates
>> Sent: 12/3/2014 9:51 AM
>> To: user@hive.apache.org
>> Subject: Re: hive tra
't be
> a need to create the tables.
>
> Alan.
>
> yogendra reddy
> November 30, 2014 at 21:19
> Hi All,
>
> I'm caught up with the error pasted below after trying to use hive
> transaction feature.I have followed this wiki to set required
>
Thanks Alan, I'm able to see TXNS table and yes I have upgraded from 0.13 to
0.14.
any idea as to wht else might cause this issue?
Thanks,
Yogendra
-Original Message-
From: "Alan Gates"
Sent: 12/3/2014 9:51 AM
To: "user@hive.apache.org"
Subject: Re: h
...@gmail.com>
November 30, 2014 at 21:19
Hi All,
I'm caught up with the error pasted below after trying to use hive
transaction feature.I have followed this wiki to set required
configuration.
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-
It is running. As I mentioned earlier only after setting transaction
properties I'm getting the error, was able to run normal hive queries if I
disable transaction related properties!
On Mon, Dec 1, 2014 at 1:52 PM, unmesha sreeveni
wrote:
>
> On Mon, Dec 1, 2014 at 12:31 PM, yogendra reddy
> w
On Mon, Dec 1, 2014 at 12:31 PM, yogendra reddy
wrote:
> WARN metastore.RetryingMetaStoreClient: MetaStoreClient lost connection.
>
It looks like hive-metastore service is not running.
Can you please check the same?
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Develop
ansaction feature.I have followed this wiki to set required
>> configuration.
>> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>>
>>
>> set hive.support.concurrency=true;
>> set hive.exec.dynamic.partition.mode=nonst
Hi All,
>
> I'm caught up with the error pasted below after trying to use hive
> transaction feature.I have followed this wiki to set required
> configuration.
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>
>
> s
Hi All,
I'm caught up with the error pasted below after trying to use hive
transaction feature.I have followed this wiki to set required
configuration.
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
set hive.support.concurrency=true
31 matches
Mail list logo
