RE: Does output directory remain in case of map/reduce task failures

2010-11-02 Thread Namit Jain
up output location in case of task failures (maybe to allow the user to debug). What is hive’s behavior? Specifically if I have a table foo and I am writing to partition datestamp=20101102 then the write would go to /user/hive/warehouse/foo/datestamp=20101102. If the task(s) writing to this

RE: Does output directory remain in case of map/reduce task failures

2010-11-02 Thread Pradeep Kamath
partition? The behavior in hadoop is to NOT clean up output location in case of task failures (maybe to allow the user to debug). What is hive's behavior? Specifically if I have a table foo and I am writing to partition datestamp=20101102 then the write would go to /user/hive/warehouse/foo/date

RE: Does output directory remain in case of map/reduce task failures

2010-11-02 Thread Namit Jain
NOT clean up output location in case of task failures (maybe to allow the user to debug). What is hive’s behavior? Specifically if I have a table foo and I am writing to partition datestamp=20101102 then the write would go to /user/hive/warehouse/foo/datestamp=20101102. If the task(s) writing

Does output directory remain in case of map/reduce task failures

2010-11-02 Thread Pradeep Kamath
to debug). What is hive's behavior? Specifically if I have a table foo and I am writing to partition datestamp=20101102 then the write would go to /user/hive/warehouse/foo/datestamp=20101102. If the task(s) writing to this fail, does hive remove this dir on exit? If it doesn't, a

Re: UDF help

2010-11-02 Thread Tim Robertson
Thanks Edward. I'll poke around there. On Tue, Nov 2, 2010 at 6:40 PM, Edward Capriolo wrote: > On Tue, Nov 2, 2010 at 12:47 PM, Tim Robertson > wrote: >> Hi all, >> >> Is the following a valid UDF please? >> >> When I run it I get the following so I presume not: >> hive> select toGoogleCoord

Re: UDF help

2010-11-02 Thread Edward Capriolo
On Tue, Nov 2, 2010 at 12:47 PM, Tim Robertson wrote: > Hi all, > > Is the following a valid UDF please? > > When I run it I get the following so I presume not: > hive> select toGoogleCoords(latitude,longitude,1) from > raw_occurrence_record limit 100; > FAILED: Error in semantic analysis: > java.

UDF help

2010-11-02 Thread Tim Robertson
Hi all, Is the following a valid UDF please? When I run it I get the following so I presume not: hive> select toGoogleCoords(latitude,longitude,1) from raw_occurrence_record limit 100; FAILED: Error in semantic analysis: java.lang.IllegalArgumentException: Error: name expected at the position 7 o