Hi Kristen, I discussed this with tez folks and they tell me the version of tez you have should work with that version of hive. It seems like the request for container could have been very large (or -1 perhaps). Can you share the hive logs as well as the application logs for this issue. Probably worth a jira ticket for tracking purposes.
Thanks Vikram. On Mon, Dec 15, 2014 at 11:01 AM, Kristen Hardwick <khardw...@spryinc.com> wrote: > Vikram, > > Thanks for the quick response. I am using the following versions: > > Hive 0.14.0.2.2.0.0 > Tez 0.5.2.2.2.0.0 > > Do you know where I can find a Hortonworks RPM for Tez 0.5.3 so that I can > upgrade manually? I used the Ambari installation from the Hortonworks site, > so I think the version conflict is built into what is available for > download. > > Thanks! > Kristen > > On Mon, Dec 15, 2014 at 1:29 PM, Vikram Dixit <vik...@hortonworks.com> > wrote: >> >> Hi Kristen, >> >> Can you share the version of tez you are using? Hive 0.14 (in HDP 2.2) >> needs tez 0.5.3 or beyond. Not sure if there was an earlier >> installation of tez that is being used here. >> >> Thanks >> Vikram. >> >> On Mon, Dec 15, 2014 at 10:13 AM, Kristen Hardwick >> <khardw...@spryinc.com> wrote: >> > Hi all, >> > >> > We recently created a new Amazon cluster with an installation of GA HDP >> > 2.2 >> > from the Hortonworks website. One of the main reasons we wanted to try >> > this >> > version was to use Hive with Tez with the new cost based optimizer. >> > However, >> > running simple queries with Tez as the execution engine does not work. >> > >> > I created a single line file, created the Hive table for it, and ran >> > this >> > query to confirm proper data load: >> > >> > select * from test; >> > >> > No issues, so I ran this query with the "mr" execution engine, and had >> > success: >> > >> > select count(*) from test; -- 1 >> > >> > Then I ran the same query with the "tez" execution engine, and had no >> > success. I got the following error: >> > >> > Status: Running (Executing on YARN cluster with App id >> > application_1418140477139_0067) >> > >> > >> > -------------------------------------------------------------------------------- >> > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED >> > >> > -------------------------------------------------------------------------------- >> > Map 1 FAILED 1 0 0 1 4 0 >> > Reducer 2 KILLED 1 0 0 1 0 1 >> > >> > -------------------------------------------------------------------------------- >> > VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 9.61 s >> > >> > -------------------------------------------------------------------------------- >> > Status: Failed >> > Vertex failed, vertexName=Map 1, >> > vertexId=vertex_1418140477139_0067_1_00, >> > diagnostics=[Task failed, taskId=task_1418140477139_0067_1_00_000000, >> > diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running >> > task:java.lang.IllegalArgumentException >> > at >> > com.google.common.base.Preconditions.checkArgument(Preconditions.java:72) >> > at >> > >> > org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:158) >> > at >> > >> > org.apache.tez.runtime.common.resources.MemoryDistributor.requestMemory(MemoryDistributor.java:97) >> > at >> > >> > org.apache.tez.runtime.api.impl.TezTaskContextImpl.requestInitialMemory(TezTaskContextImpl.java:192) >> > at >> > >> > org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.initialize(OrderedPartitionedKVOutput.java:95) >> > >> > Have any of you seen this issue before? That "requestInitialMemory" >> > function >> > failure combined with the "IllegalArgumentException" message made me >> > think >> > that maybe Tez was asking for a container that was too big...but I >> > didn't >> > see anything odd in the properties. Then again, I don't have a lot of >> > insight into what this new version of Tez requires. >> > >> > Any help is appreciated! Please let me know if you need additional >> > information. >> > Kristen Hardwick >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified >> that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> immediately >> and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
