Hey sorry, I was moving house all evening and driving around town :)
I did not automate my unit tests, I created a small frame app to test each
function and make sure it responded appropriately. Good to know junit will do
it.
For your other question, did you include a call to init() in your ud
I'll just keep responding to myself. ;)
I ended up figuring out how to do it. I just used junit and called init,
iterate, terminatePartial, etc from inside the unit test. After knowing a
typical flow of function calls (as I mentioned below), the main other gotcha is
making sure to have a new UD
I'm looking for something like this, but for a UDAF instead of a UDF:
http://svn.apache.org/repos/asf/hive/branches/branch-0.7/ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFDateDiff.java
-Aurora
On Mar 11, 2011, at 4:44 PM, Aurora Skarra-Gallagher wrote:
> Hi,
>
> Did you actually call those
Anyone know the answer to this?
Thanks,
Aurora
On Feb 15, 2011, at 8:53 AM, Aurora Skarra-Gallagher wrote:
> Hi,
>
> I wrote a simple UDAF for Hive 0.6 and I had to include null checks in
> terminatePartial even though the object should never be null if init is
> always called before terminat
Hi,
Did you actually call those functions directly from your unit tests? I'm
looking for examples of that working, but all I see reference to are tests to
make sure the query produces the expected output (rather than directly testing
the UDAF).
-Aurora
On Mar 11, 2011, at 3:44 PM, Christopher
Awesome, awesome. That's what I had pieced together from Steve and Ed's
emails. Glad to get confirmation on it.
Its also what I did for my unit testing. I also called everything with null
arguments to make sure those got handled gracefully.
Pat
-Original Message-
From: Aurora Skarra
Hadoop: The Definitive Guide has a good section on this. Chapter 12: Hive: User
Defined Functions. It has a diagram that shows how things are called and when.
The example I'm looking at shows this sequence:
(first instance)
init()
iterate(1)
iterate(2)
iterate(3)
terminatePartial()
(second inst
Hi,
Does anyone have experience/examples on unit testing UDAFs? I'm thinking I need
to test by calling some sequence like: init, iterate, iterate ...,
terminatePartial, init, merge, terminate. I'm not sure though. Any advice would
be appreciated. Thanks!
Thanks,
Aurora
Ahh, perfect. The docs don't agree terribly well but the case study is great.
The context for when merge() gets called was not clear to me.
Thanks guys!
Pat
-Original Message-
From: Steven Wong [mailto:sw...@netflix.com]
Sent: Thursday, March 10, 2011 6:24 PM
To: user@hive.apache.org
Hi hadoop n00b:
Jasperforge has some connectors for Hadoop. I have not tried them out yet as I
am new to the JasperServer platform. It is on my todo list. It's presence on
JasperForge suggests you can run it with the community edition of their
JasperServer (and enterprise too).
JasperServer
,on top of
>> which hive runs.
>> 3. I am not too familiar with map-reduce,however to the best of my
>> knowledge all the configurations in the jobtrackers as well as core-
>> utilization were the default ones.
>> 4. No swapping occurs( ~200mb remains free all the time)
>>
gurations in the jobtrackers as well as core-
> utilization were the default ones.
> 4. No swapping occurs( ~200mb remains free all the time)
>
> For more information I give below the output of a sample query:
>
> hive> select count (1) from searchlogs where days=20110311;
> select count (1) from searchlogs where days=20110311;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=
In order to limit the maximum num
13 matches
Mail list logo
