Set -m flag to 1 programmatically On Oct 1, 2014 8:13 PM, "Thiago Henrique dos Santos Bento" < thiago.be...@b2wdigital.com> wrote:
> Hi! > > > > I'm trying to run a simple join with two tables stored as ORC File. (the > same query runs normally without ORC File) > > I got this error while map-reduce job: > > Driver returned: 2. Errors: OK > > Query ID = hue_20141001071313_33e31d68-3b4a-49d5-acc0-30c0a42276ab > > Total jobs = 1 > > Stage-1 is selected by condition resolver. > > Launching Job 1 out of 1 > > Number of reduce tasks not specified. Estimated from input data size: 12 > > In order to change the average load for a reducer (in bytes): > > set hive.exec.reducers.bytes.per.reducer=<number> > > In order to limit the maximum number of reducers: > > set hive.exec.reducers.max=<number> > > In order to set a constant number of reducers: > > set mapreduce.job.reduces=<number> > > Starting Job = job_1411750973876_0099, Tracking URL = > http://arnaldo02:8088/proxy/application_1411750973876_0099/ > > Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1411750973876_0099 > > Hadoop job information for Stage-1: number of mappers: 42; number of > reducers: 12 > > 2014-10-01 07:13:51,965 Stage-1 map = 0%, reduce = 0% > > 2014-10-01 07:14:18,882 Stage-1 map = 2%, reduce = 0%, Cumulative CPU > 37.32 sec > > 2014-10-01 07:14:19,990 Stage-1 map = 3%, reduce = 0%, Cumulative CPU > 40.31 sec > > 2014-10-01 07:14:21,100 Stage-1 map = 5%, reduce = 0%, Cumulative CPU > 43.33 sec > > 2014-10-01 07:14:25,544 Stage-1 map = 7%, reduce = 0%, Cumulative CPU > 57.49 sec > > 2014-10-01 07:14:39,996 Stage-1 map = 8%, reduce = 0%, Cumulative CPU > 73.08 sec > > 2014-10-01 07:14:43,338 Stage-1 map = 9%, reduce = 0%, Cumulative CPU > 76.76 sec > > 2014-10-01 07:14:48,882 Stage-1 map = 11%, reduce = 0%, Cumulative CPU > 92.88 sec > > 2014-10-01 07:14:51,107 Stage-1 map = 12%, reduce = 0%, Cumulative CPU > 100.93 sec > > 2014-10-01 07:14:52,212 Stage-1 map = 13%, reduce = 0%, Cumulative CPU > 106.04 sec > > 2014-10-01 07:14:54,426 Stage-1 map = 14%, reduce = 0%, Cumulative CPU > 111.02 sec > > 2014-10-01 07:14:57,751 Stage-1 map = 15%, reduce = 0%, Cumulative CPU > 126.41 sec > > 2014-10-01 07:15:05,507 Stage-1 map = 17%, reduce = 0%, Cumulative CPU > 135.71 sec > > 2014-10-01 07:15:10,023 Stage-1 map = 18%, reduce = 0%, Cumulative CPU > 146.69 sec > > 2014-10-01 07:15:19,987 Stage-1 map = 19%, reduce = 0%, Cumulative CPU > 156.43 sec > > 2014-10-01 07:15:29,957 Stage-1 map = 19%, reduce = 1%, Cumulative CPU > 168.3 sec > > 2014-10-01 07:15:37,703 Stage-1 map = 21%, reduce = 1%, Cumulative CPU > 179.77 sec > > 2014-10-01 07:15:45,511 Stage-1 map = 23%, reduce = 1%, Cumulative CPU > 192.87 sec > > 2014-10-01 07:15:49,948 Stage-1 map = 25%, reduce = 1%, Cumulative CPU > 204.72 sec > > 2014-10-01 07:15:53,287 Stage-1 map = 26%, reduce = 1%, Cumulative CPU > 211.83 sec > > 2014-10-01 07:16:05,456 Stage-1 map = 28%, reduce = 1%, Cumulative CPU > 231.45 sec > > 2014-10-01 07:16:07,664 Stage-1 map = 29%, reduce = 1%, Cumulative CPU > 233.46 sec > > 2014-10-01 07:16:09,876 Stage-1 map = 29%, reduce = 2%, Cumulative CPU > 233.74 sec > > 2014-10-01 07:16:15,408 Stage-1 map = 32%, reduce = 2%, Cumulative CPU > 255.51 sec > > 2014-10-01 07:16:18,727 Stage-1 map = 33%, reduce = 2%, Cumulative CPU > 258.57 sec > > 2014-10-01 07:16:19,830 Stage-1 map = 33%, reduce = 3%, Cumulative CPU > 261.95 sec > > 2014-10-01 07:16:35,288 Stage-1 map = 35%, reduce = 3%, Cumulative CPU > 288.59 sec > > 2014-10-01 07:16:40,806 Stage-1 map = 37%, reduce = 3%, Cumulative CPU > 298.89 sec > > 2014-10-01 07:16:45,278 Stage-1 map = 38%, reduce = 3%, Cumulative CPU > 306.72 sec > > 2014-10-01 07:16:58,531 Stage-1 map = 38%, reduce = 4%, Cumulative CPU > 312.76 sec > > 2014-10-01 07:17:02,945 Stage-1 map = 40%, reduce = 4%, Cumulative CPU > 321.87 sec > > 2014-10-01 07:17:22,828 Stage-1 map = 42%, reduce = 4%, Cumulative CPU > 336.71 sec > > 2014-10-01 07:17:28,341 Stage-1 map = 43%, reduce = 5%, Cumulative CPU > 343.66 sec > > 2014-10-01 07:17:39,370 Stage-1 map = 44%, reduce = 5%, Cumulative CPU > 350.7 sec > > 2014-10-01 07:17:51,541 Stage-1 map = 47%, reduce = 5%, Cumulative CPU > 365.91 sec > > 2014-10-01 07:18:08,097 Stage-1 map = 49%, reduce = 5%, Cumulative CPU > 375.35 sec > > 2014-10-01 07:18:22,452 Stage-1 map = 49%, reduce = 7%, Cumulative CPU > 383.34 sec > > 2014-10-01 07:18:25,756 Stage-1 map = 50%, reduce = 7%, Cumulative CPU > 383.69 sec > > 2014-10-01 07:19:26,488 Stage-1 map = 50%, reduce = 7%, Cumulative CPU > 389.38 sec > > 2014-10-01 07:19:50,793 Stage-1 map = 52%, reduce = 7%, Cumulative CPU > 391.28 sec > > 2014-10-01 07:19:58,551 Stage-1 map = 50%, reduce = 7%, Cumulative CPU > 386.39 sec > > 2014-10-01 07:20:50,430 Stage-1 map = 52%, reduce = 7%, Cumulative CPU > 393.79 sec > > 2014-10-01 07:21:51,114 Stage-1 map = 52%, reduce = 7%, Cumulative CPU > 411.94 sec > > 2014-10-01 07:22:19,773 Stage-1 map = 54%, reduce = 7%, Cumulative CPU > 417.31 sec > > 2014-10-01 07:23:20,493 Stage-1 map = 54%, reduce = 7%, Cumulative CPU > 423.21 sec > > 2014-10-01 07:24:21,168 Stage-1 map = 54%, reduce = 7%, Cumulative CPU > 425.94 sec > > 2014-10-01 07:25:21,814 Stage-1 map = 54%, reduce = 7%, Cumulative CPU > 441.2 sec > > 2014-10-01 07:25:36,145 Stage-1 map = 55%, reduce = 7%, Cumulative CPU > 442.7 sec > > 2014-10-01 07:26:36,837 Stage-1 map = 55%, reduce = 7%, Cumulative CPU > 448.75 sec > > 2014-10-01 07:26:40,140 Stage-1 map = 55%, reduce = 8%, Cumulative CPU > 449.35 sec > > 2014-10-01 07:27:11,064 Stage-1 map = 56%, reduce = 8%, Cumulative CPU > 455.24 sec > > 2014-10-01 07:27:19,871 Stage-1 map = 57%, reduce = 8%, Cumulative CPU > 460.35 sec > > 2014-10-01 07:27:56,277 Stage-1 map = 58%, reduce = 8%, Cumulative CPU > 481.34 sec > > 2014-10-01 07:28:40,332 Stage-1 map = 59%, reduce = 8%, Cumulative CPU > 504.81 sec > > 2014-10-01 07:29:40,957 Stage-1 map = 59%, reduce = 8%, Cumulative CPU > 510.71 sec > > 2014-10-01 07:30:41,638 Stage-1 map = 59%, reduce = 8%, Cumulative CPU > 513.74 sec > > 2014-10-01 07:31:00,419 Stage-1 map = 60%, reduce = 8%, Cumulative CPU > 514.6 sec > > 2014-10-01 07:32:01,081 Stage-1 map = 60%, reduce = 8%, Cumulative CPU > 518.18 sec > > 2014-10-01 07:32:50,724 Stage-1 map = 62%, reduce = 8%, Cumulative CPU > 520.43 sec > > 2014-10-01 07:33:00,648 Stage-1 map = 60%, reduce = 8%, Cumulative CPU > 521.71 sec > > 2014-10-01 07:33:14,974 Stage-1 map = 100%, reduce = 100%, Cumulative CPU > 459.27 sec > > MapReduce Total cumulative CPU time: 7 minutes 39 seconds 270 msec > > Ended Job = job_1411750973876_0099 with errors > > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.mr.MapRedTask > > MapReduce Jobs Launched: > > Job 0: Map: 42 Reduce: 12 Cumulative CPU: 459.27 sec HDFS Read: > 1989875843 HDFS Write: 0 FAIL > > Total MapReduce CPU Time Spent: 7 minutes 39 seconds 270 msec > > > > > > Any idea? > > > > Tks, > > *Thiago Henrique S. Bento* > > B2W Digital / B2W Labs | BIT SP > > thiago.be...@b2wdigital.com > > Tel.: +55 (11) 3454-3024 > > > ............................................................................................................................................................. > > *B**2**W DIGITAL* #Americanas.com <http://americanas.com/> #Shoptime.com > <http://shoptime.com/> #Submarino.com <http://submarino.com/> # > SouBarato.com <http://soubarato.com/> > > > > > ------------------------------ > > Esta mensagem pode conter informações confidenciais e somente o indivíduo > ou entidade a quem foi destinada pode utilizá-la. A transmissão incorreta > da mensagem não acarreta a perda de sua confidencialidade. Caso esta > mensagem tenha sido recebida por engano, solicitamos que o fato seja > comunicado ao remetente e que a mensagem seja eliminada de seu sistema > imediatamente. É vedado a qualquer pessoa que não seja o destinatário usar, > revelar, distribuir ou copiar qualquer parte desta mensagem. Ambiente de > comunicação sujeito a monitoramento. > > This message may include confidential information and only the intended > addresses have the right to use it as is, or any part of it. A wrong > transmission does not break its confidentiality. If you've received it > because of a mistake or erroneous transmission, please notify the sender > and delete it from your system immediately. This communication environment > is controlled and monitored. > > B2W Digital > > >
