Re: Loading a Hive table simultaneously from 2 different sources

2013-01-24 Thread Dean Wampler
You'll face all the usual concurrency synchronization risks if you're updating the same "place" concurrently. One thing to keep in mind; it's all just HDFS under the hood. That pretty much tells you everything you need to know. Yes, there's also the metadata. So, one way to update a partition direc

Re: Loading a Hive table simultaneously from 2 different sources

2013-01-24 Thread Krishnan K
Hi Edward, All, Thanks for the quick reply! We are using dynamic partitions - so unable to say to which partition each record goes. We dont have much control here. Is there any properties that can be set ? I'm a bit doubtful here - is it because of the lock acquired on the table ? Regards, Kris

RE: Loading a Hive table simultaneously from 2 different sources

2013-01-24 Thread Bennie Schut
. From: Edward Capriolo [mailto:edlinuxg...@gmail.com] Sent: Thursday, January 24, 2013 3:52 PM To: user@hive.apache.org Subject: Re: Loading a Hive table simultaneously from 2 different sources Partition the table and load the data into different partitions. That or build the data outside he tab

Re: Loading a Hive table simultaneously from 2 different sources

2013-01-24 Thread Edward Capriolo
Partition the table and load the data into different partitions. That or build the data outside he table and then use scripting to move the data in using LOAD DATA INPATH or copying. On Thu, Jan 24, 2013 at 9:44 AM, Krishnan K wrote: > Hi All, > > Could you please let me know what would happen i