Thank you. Are there tools for parsing the Hive logs for errors? If not,
can you talk about the strategy used at Facebook to deal with detection and
resolution of MR errors?
Perhaps I can write a script to identify errors. First I have to solve the
mystery of why there are no logs on my hadoop
Hive queries are compiled to different types tasks (MapReduce, MoveTask, etc),
so a successful MR task as indicated in the JT doesn't mean the whole query
succeeded. So you need to examine the status of the hive query to see if it
succeeded or not. You can also check the hive's log file under
/
I've been using Hive in production for two months now. We're mainly using
it for processing server logs, about 1-2GB per day (2-2.5 million
requests). Typically we import a day's worth of logs at once. That said,
sometimes we decide to tweak a calculated column. When that happens, we
modify our
you could always have another sub partition under the daily partition. This sub
partition could be the timestamp on when you did the load. So when you run the
statement you would create a new sub partition within the date partition and in
effect you end up doing an append to the Hive partition.
With streaming, UDF or UDTFs you would get almost any kind of control flow you
want without having those features implemented in Hive proper. For udf, udaf or
udtf you use java for implementation. In streaming you can use any language of
your choice. Not sure if this addresses stuff?
Ashish
On
Thank you very much!
I am going to use your approach at once.
Hope I can get some luck. :-D
Best regards.
2011/5/11 Jov :
> I do it like this:
>
> 1.down load the hive src
> 2.compile hive in eclipse from my windows,which can connect the net.it
> will down load all necessories to src/build dir
Hi experts,
I got to know that hive has no stored procedures.but this can be done in
Hive with the help of streaming and functions.Is there any way to implement
*Declare,Conditional handling statements,Control statements* in hive.As of
now i'm looking into Hive UDF's and some stored procedures in M
I do it like this:
1.down load the hive src
2.compile hive in eclipse from my windows,which can connect the net.it
will down load all necessories to src/build dir
3.in linux,mkdir src/build;copy the src/build/hadoopcore,src/ivy from
windows to linux
4.edit the src/build-common.xml,make the target
Hello everyone.
I am a student who begins to do some research on Hive project.
I think the first step is to have a successful build of Hive.
I have downloaded the Hive source code, uploaded to the Linux machine,
unzipped and entered the source directory.
Then I run the "ant" command and got a error
