Hi all, We recently created a new Amazon cluster with an installation of GA HDP 2.2 from the Hortonworks website. One of the main reasons we wanted to try this version was to use Hive with Tez with the new cost based optimizer. However, running simple queries with Tez as the execution engine does not work.
I created a single line file, created the Hive table for it, and ran this query to confirm proper data load: select * from test; No issues, so I ran this query with the "mr" execution engine, and had success: select count(*) from test; -- 1 Then I ran the same query with the "tez" execution engine, and had no success. I got the following error: Status: Running (Executing on YARN cluster with App id application_1418140477139_0067) -------------------------------------------------------------------------------- VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED -------------------------------------------------------------------------------- Map 1 FAILED 1 0 0 1 4 0 Reducer 2 KILLED 1 0 0 1 0 1 -------------------------------------------------------------------------------- VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 9.61 s -------------------------------------------------------------------------------- Status: Failed Vertex failed, vertexName=Map 1, vertexId=vertex_1418140477139_0067_1_00, diagnostics=[Task failed, taskId=task_1418140477139_0067_1_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.IllegalArgumentException at com.google.common.base.Preconditions.checkArgument(Preconditions.java:72) at org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:158) at org.apache.tez.runtime.common.resources.MemoryDistributor.requestMemory(MemoryDistributor.java:97) at org.apache.tez.runtime.api.impl.TezTaskContextImpl.requestInitialMemory(TezTaskContextImpl.java:192) at org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.initialize(OrderedPartitionedKVOutput.java:95) Have any of you seen this issue before? That "requestInitialMemory" function failure combined with the "IllegalArgumentException" message made me think that maybe Tez was asking for a container that was too big...but I didn't see anything odd in the properties. Then again, I don't have a lot of insight into what this new version of Tez requires. Any help is appreciated! Please let me know if you need additional information. Kristen Hardwick
