can I use hive dynamic partition while loading data into tables?

2011-04-14 Thread Erix Yao
hi,all The dynamic partition function is amazing ,but only works in insert clause. Can I use it while loading data into table? For example: load data LOAD DATA LOCAL INPATH `/tmp/pv_2008-06-08_us.txt` INTO TABLE page_view PARTITION(date='2008-06-08', country='US', type); type is the dynam

Hive map task attemps 33+

2011-04-14 Thread Jasper Knulst
Hi, Today I had to kill quite a large hive generated MR job. The progress on the mappers was reversed halfway (so actually declining). When I got to the local mapped logs from the TT I saw that there were up to 33 task attempts on the mappers that stalled and it looked like this would go on foreve

Using hive as a mapreduce backend

2011-04-14 Thread John Vines
Our environment is heavy into storing data in hive. I find myself currently working on something that it outside the scope though. I have a mapreduce written, but it requires a lot of direct user inputs for information that could easily be scraped from Hive. That said, when I query hive for extende