Hello,
I have been trying to optimize one of my longer running queries using a
MAPJOIN hint. The query is fairly complex and it joins my base table (1+
billion rows) with multiple metadata tables (which are relatively small in
size).
I already use a STREAMTABLE hint for my large table and have pr
When the following query was run with mapred.job.reuse.jvm.num.tasks=20, some
of the map tasks failed with "Error: Java heap space", causing the job to fail.
After changing to mapred.job.reuse.jvm.num.tasks=1, the job succeeded.
FROM (
FROM intable1
SELECT acct_id, esn) b
JOIN (
FROM
Right now Hive's TRANSFORM mechanism does not allow you set counters.
There's a ticket open for adding this feature:
https://issues.apache.org/jira/browse/HIVE-1649
Thanks.
Carl
On Thu, Apr 7, 2011 at 10:35 AM, Avram Aelony wrote:
>
> Can you try using using streaming from Hive?
>
>htt
Can you try using using streaming from Hive?
http://wiki.apache.org/hadoop/Hive/LanguageManual/Transform
Cheers,
~Avram
On Apr 7, 2011, at 10:28 AM, Henry Liu wrote:
>
> I'd like to report some counters in my customized map and reduce ruby scripts
> for hive, just like hadoop strea
I'd like to report some counters in my customized map and reduce ruby scripts
for hive, just like hadoop streaming does by writing to stderr.
Such as.
#increment hadoop counters
$stderr.puts("reporter:counter:Impression,Total,1")
if first_click == 0 && click_details.empty?
Thanks Edward,
That worked.
2011/4/7 Edward Capriolo
> On Thu, Apr 7, 2011 at 8:38 AM, Jasper Knulst
> wrote:
> > Hi,
> > How would I set the field separator for Hive output to files?
> > I see that the default is a space (or tab, don't know exactly) but I
> would
> > like to use another ch
On Thu, Apr 7, 2011 at 8:38 AM, Jasper Knulst wrote:
> Hi,
> How would I set the field separator for Hive output to files?
> I see that the default is a space (or tab, don't know exactly) but I would
> like to use another character to facilitate loading of the output in another
> system.
> I alre
Hi,
How would I set the field separator for Hive output to files?
I see that the default is a space (or tab, don't know exactly) but I would
like to use another character to facilitate loading of the output in another
system.
I already tried
set mapred.textoutputformat.separator='~';
But this
Well, I'm new here but I can point you to the docs as well as old
timers probably. First, I think the developers would prefer that you
only direct questions like this to the users list, not to both. Other
answers are inline below:
On Thu, Apr 7, 2011 at 12:00 AM, komara nagarjuna
wrote:
>
> In
