Re: Unit testing Hive script

2011-02-18 Thread Andrew Wilson
Hi Radek, I've been using the MiniMRCluster and MiniDFSCluster to run unit tests locally. That has been giving me decent cycle time. I have fixture tables in my test/resources which I can load into the MiniHiveCluster as part of test setup. I loosely based my code on QTestUtil in the Hive trunk

Re: Unit testing Hive script

2011-02-18 Thread Kirk True
Hi Radek, I'm actually in the process of running the map-join unit tests against EMR as we speak. It's possible but dog slow :) Thanks, Kirk On 2/18/11 11:09 AM, Edward Capriolo wrote: On Fri, Feb 18, 2011 at 6:58 AM, Radek Maciaszek wrote: Hello, I was wondering if anyone managed to unit

Re: Unit testing Hive script

2011-02-18 Thread Edward Capriolo
On Fri, Feb 18, 2011 at 6:58 AM, Radek Maciaszek wrote: > Hello, > I was wondering if anyone managed to unit test Hive scripts and share > his/her experience? My first thought was to prepare sample data, run hive > scripts in order to generate output and then compare the generated output > with th

Unit testing Hive script

2011-02-18 Thread Radek Maciaszek
Hello, I was wondering if anyone managed to unit test Hive scripts and share his/her experience? My first thought was to prepare sample data, run hive scripts in order to generate output and then compare the generated output with the expected output. Sounds fairly simple but it may be a bit compli