Re: Incremental import from PostgreSQL to Hive having issues

2012-04-15 Thread Nitin Pawar
best way to solve this is load the data in different partition each time you load the data. (depending on the data you can put the data partitioned by date or data-hour combination frequency on which you load the data) I am not sure how you are installing sqoop. If you are using yum on redhat, the

Re: Incremental import from PostgreSQL to Hive having issues

2012-04-15 Thread Roshan Pradeep
Hi Nitin Thanks for your reply. I am using sqoop *1.4.1-incubating* version. In the sqoop releases download page the is no such version you are referring. Please correct me if I am wrong. Delete the warehouse folder and import is working fine, but my tables having GB of data, so every time delet

Re: Case Studies for 'Programming Hive' book from O'Reilly

2012-04-15 Thread Tim Robertson
Hi Jason, I work for an international organization involved in the mobilization of biodiversity data (specifically we are dealing a lot with observations of species) so think of it as a lot of point based information with metadata tags. We have built an Oozie workflow that uses Sqoop to suck in a