Greetings Thanx Sure can. Below is from HIVE CLI
hive> select count(distinct warctype) from commoncrawl18 where warctype='warcinfo'; Query ID = jmill383_20160518143715_34041e3e-713b-4e35-ae86-a88498192ab1 Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks determined at compile time: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapreduce.job.reduces=<number> Starting Job = job_1463594979064_0003, Tracking URL = http://starchild:8088/proxy/application_1463594979064_0003/ Kill Command = /opt/hadoop/bin/hadoop job -kill job_1463594979064_0003 Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0 2016-05-18 14:37:19,794 Stage-1 map = 0%, reduce = 0% Ended Job = job_1463594979064_0003 with errors Error during job, obtaining debugging information... Job Tracking URL: http://starchild:8088/cluster/app/application_1463594979064_0003 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched: Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 0 msec hive> This one is from cascading-hive [jmill383@starchild demo]$ /opt/hadoop/bin/hadoop jar build/libs/cascading-hive-demo-1.0.jar cascading.hive.HiveDemo SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hive/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 16/05/18 14:40:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 16/05/18 14:40:20 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:20 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:21 WARN DataNucleus.General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/tmp/hadoop-unjar5875988559818865999/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/usr/local/hive/lib/datanucleus-api-jdo-3.2.6.jar." 16/05/18 14:40:21 WARN DataNucleus.General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/tmp/hadoop-unjar5875988559818865999/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/usr/local/hive/lib/datanucleus-core-3.2.10.jar." 16/05/18 14:40:21 WARN DataNucleus.General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/tmp/hadoop-unjar5875988559818865999/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/usr/local/hive/lib/datanucleus-rdbms-3.2.9.jar." 16/05/18 14:40:21 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored 16/05/18 14:40:21 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 16/05/18 14:40:22 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 16/05/18 14:40:23 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 16/05/18 14:40:23 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 16/05/18 14:40:23 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 16/05/18 14:40:23 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 16/05/18 14:40:24 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:24 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:24 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:24 INFO metastore.HiveMetaStore: Added admin role in metastore 16/05/18 14:40:24 INFO metastore.HiveMetaStore: Added public role in metastore 16/05/18 14:40:24 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=dual 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Shutting down the object store... 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Shutting down the object store... 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Metastore shutdown complete. 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Metastore shutdown complete. 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=dual 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:24 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:24 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:24 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:24 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Shutting down the object store... 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Shutting down the object store... 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Metastore shutdown complete. 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Metastore shutdown complete. 16/05/18 14:40:24 INFO property.AppProps: using app.id: 954F6CFECF794BC191AB3296A6FAC1F5 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=keyvalue 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=keyvalue 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:24 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:24 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:24 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:24 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Shutting down the object store... 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Shutting down the object store... 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Metastore shutdown complete. 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Metastore shutdown complete. 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=keyvalue 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=keyvalue 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:24 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:24 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:24 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:24 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Shutting down the object store... 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Shutting down the object store... 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Metastore shutdown complete. 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Metastore shutdown complete. 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=keyvalue2 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=keyvalue2 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:24 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:24 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:24 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:24 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Shutting down the object store... 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Shutting down the object store... 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Metastore shutdown complete. 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Metastore shutdown complete. 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=keyvalue2 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=keyvalue2 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:24 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:24 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:24 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:24 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Shutting down the object store... 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Shutting down the object store... 16/05/18 14:40:24 INFO metastore.HiveMetaStore: 0: Metastore shutdown complete. 16/05/18 14:40:24 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Metastore shutdown complete. 16/05/18 14:40:24 INFO util.Util: resolving application jar from found main method on: cascading.hive.HiveDemo 16/05/18 14:40:24 INFO planner.HadoopPlanner: using application jar: /home/jmill383/cascading-hive/demo/build/libs/cascading-hive-demo-1.0.jar 16/05/18 14:40:25 INFO flow.Flow: [uppercase kv -> kv2 ] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.050 16/05/18 14:40:25 INFO flow.Flow: [uppercase kv -> kv2 ] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1 16/05/18 14:40:25 INFO flow.Flow: [uppercase kv -> kv2 ] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes' 16/05/18 14:40:25 INFO Configuration.deprecation: mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used 16/05/18 14:40:25 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir 16/05/18 14:40:25 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 16/05/18 14:40:25 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir 16/05/18 14:40:25 INFO Configuration.deprecation: mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress 16/05/18 14:40:25 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class 16/05/18 14:40:25 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class 16/05/18 14:40:25 INFO util.Version: Concurrent, Inc - Cascading 3.1.0-wip-57 16/05/18 14:40:25 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] starting 16/05/18 14:40:25 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] parallel execution of flows is enabled: false 16/05/18 14:40:25 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] executing total flows: 3 16/05/18 14:40:25 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] allocating management threads: 1 16/05/18 14:40:25 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] starting flow: load data into dual 16/05/18 14:40:25 INFO flow.Flow: [load data into dual] at least one sink is marked for delete 16/05/18 14:40:25 INFO flow.Flow: [load data into dual] sink oldest modified date: Wed Dec 31 18:59:59 EST 1969 16/05/18 14:40:25 INFO metastore.HiveMetaStore: 1: get_table : db=default tbl=dual 16/05/18 14:40:25 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:25 INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:25 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:25 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:25 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:25 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:25 INFO hive.HiveTap: strict mode: comparing existing hive table with table descriptor 16/05/18 14:40:25 INFO metastore.HiveMetaStore: 1: Shutting down the object store... 16/05/18 14:40:25 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Shutting down the object store... 16/05/18 14:40:25 INFO metastore.HiveMetaStore: 1: Metastore shutdown complete. 16/05/18 14:40:25 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Metastore shutdown complete. 16/05/18 14:40:25 INFO metastore.HiveMetaStore: 2: get_all_databases 16/05/18 14:40:25 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_all_databases 16/05/18 14:40:25 INFO metastore.HiveMetaStore: 2: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:25 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:25 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:25 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:25 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:25 INFO metastore.HiveMetaStore: 2: get_functions: db=default pat=* 16/05/18 14:40:25 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_functions: db=default pat=* 16/05/18 14:40:25 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table. 16/05/18 14:40:25 INFO session.SessionState: Created local directory: /tmp/e9c0df1e-0647-47df-ad9b-ddc1dcdb6054_resources 16/05/18 14:40:25 INFO session.SessionState: Created HDFS directory: /tmp/hive/jmill383/e9c0df1e-0647-47df-ad9b-ddc1dcdb6054 16/05/18 14:40:25 INFO session.SessionState: Created local directory: /tmp/jmill383/e9c0df1e-0647-47df-ad9b-ddc1dcdb6054 16/05/18 14:40:25 INFO session.SessionState: Created HDFS directory: /tmp/hive/jmill383/e9c0df1e-0647-47df-ad9b-ddc1dcdb6054/_tmp_space.db 16/05/18 14:40:25 INFO hive.HiveQueryRunner: running hive query: 'load data local inpath 'file:///home/jmill383/cascading-hive/demo/src/main/resources/data.txt' overwrite into table dual' 16/05/18 14:40:25 INFO log.PerfLogger: <PERFLOG method=Driver.run from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:25 INFO log.PerfLogger: <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:25 INFO log.PerfLogger: <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:25 INFO log.PerfLogger: <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:25 INFO parse.ParseDriver: Parsing command: load data local inpath 'file:///home/jmill383/cascading-hive/demo/src/main/resources/data.txt' overwrite into table dual 16/05/18 14:40:26 INFO parse.ParseDriver: Parse Completed 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=parse start=1463596825623 end=1463596826130 duration=507 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 2: get_table : db=default tbl=dual 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:26 INFO ql.Driver: Semantic Analysis Completed 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=semanticAnalyze start=1463596826132 end=1463596826291 duration=159 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:null, properties:null) 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=compile start=1463596825602 end=1463596826295 duration=693 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO ql.Driver: Starting command(queryId=jmill383_20160518144025_dbe97c43-6e94-43a9-bf17-14dd2d88f490): load data local inpath 'file:///home/jmill383/cascading-hive/demo/src/main/resources/data.txt' overwrite into table dual 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=TimeToSubmit start=1463596825602 end=1463596826298 duration=696 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=task.MOVE.Stage-0 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO ql.Driver: Starting task [Stage-0:MOVE] in serial mode Loading data to table default.dual 16/05/18 14:40:26 INFO exec.Task: Loading data to table default.dual from file:/home/jmill383/cascading-hive/demo/src/main/resources/data.txt 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 2: get_table : db=default tbl=dual 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 2: get_table : db=default tbl=dual 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:26 INFO common.FileUtils: deleting hdfs://localhost:8025/user/hive/warehouse/dual/data.txt 16/05/18 14:40:26 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes. 16/05/18 14:40:26 INFO metadata.Hive: Replacing src:file:/home/jmill383/cascading-hive/demo/src/main/resources/data.txt, dest: hdfs://localhost:8025/user/hive/warehouse/dual/data.txt, Status:true 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 2: alter_table: db=default tbl=dual newtbl=dual 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=alter_table: db=default tbl=dual newtbl=dual 16/05/18 14:40:26 INFO hive.log: Updating table stats fast for dual 16/05/18 14:40:26 INFO hive.log: Updated size of table dual to 2 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=task.STATS.Stage-1 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO ql.Driver: Starting task [Stage-1:STATS] in serial mode 16/05/18 14:40:26 INFO exec.StatsTask: Executing stats task 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 2: get_table : db=default tbl=dual 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 2: get_table : db=default tbl=dual 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 2: alter_table: db=default tbl=dual newtbl=dual 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=alter_table: db=default tbl=dual newtbl=dual 16/05/18 14:40:26 INFO hive.log: Updating table stats fast for dual 16/05/18 14:40:26 INFO hive.log: Updated size of table dual to 2 Table default.dual stats: [numFiles=1, numRows=0, totalSize=2, rawDataSize=0] 16/05/18 14:40:26 INFO exec.Task: Table default.dual stats: [numFiles=1, numRows=0, totalSize=2, rawDataSize=0] 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=runTasks start=1463596826298 end=1463596826726 duration=428 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=Driver.execute start=1463596826296 end=1463596826726 duration=430 from=org.apache.hadoop.hive.ql.Driver> OK 16/05/18 14:40:26 INFO ql.Driver: OK 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=releaseLocks start=1463596826727 end=1463596826727 duration=0 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=Driver.run start=1463596825602 end=1463596826727 duration=1125 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=releaseLocks start=1463596826728 end=1463596826728 duration=0 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] completed flow: load data into dual 16/05/18 14:40:26 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] starting flow: select data from dual into keyvalue 16/05/18 14:40:26 INFO flow.Flow: [select data from dual ...] at least one sink is marked for delete 16/05/18 14:40:26 INFO flow.Flow: [select data from dual ...] sink oldest modified date: Wed Dec 31 18:59:59 EST 1969 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 1: get_table : db=default tbl=keyvalue 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=keyvalue 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:26 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:26 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:26 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:26 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:26 INFO hive.HiveTap: strict mode: comparing existing hive table with table descriptor 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 1: Shutting down the object store... 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Shutting down the object store... 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 1: Metastore shutdown complete. 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=Metastore shutdown complete. 16/05/18 14:40:26 INFO session.SessionState: Created local directory: /tmp/b51c81d8-23b9-49ea-b012-92f81bc1c0ce_resources 16/05/18 14:40:26 INFO session.SessionState: Created HDFS directory: /tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce 16/05/18 14:40:26 INFO session.SessionState: Created local directory: /tmp/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce 16/05/18 14:40:26 INFO session.SessionState: Created HDFS directory: /tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/_tmp_space.db 16/05/18 14:40:26 INFO hive.HiveQueryRunner: running hive query: 'insert overwrite table keyvalue select 'Hello' as key, 'hive!' as value from dual' 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=Driver.run from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=compile from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=parse from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO parse.ParseDriver: Parsing command: insert overwrite table keyvalue select 'Hello' as key, 'hive!' as value from dual 16/05/18 14:40:26 INFO parse.ParseDriver: Parse Completed 16/05/18 14:40:26 INFO log.PerfLogger: </PERFLOG method=parse start=1463596826826 end=1463596826833 duration=7 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO log.PerfLogger: <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:26 INFO parse.CalcitePlanner: Starting Semantic Analysis 16/05/18 14:40:26 INFO parse.CalcitePlanner: Completed phase 1 of Semantic Analysis 16/05/18 14:40:26 INFO parse.CalcitePlanner: Get metadata for source tables 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 3: get_table : db=default tbl=dual 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=dual 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 3: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 16/05/18 14:40:26 INFO metastore.ObjectStore: ObjectStore, initialize called 16/05/18 14:40:26 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 16/05/18 14:40:26 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 16/05/18 14:40:26 INFO metastore.ObjectStore: Initialized ObjectStore 16/05/18 14:40:26 INFO parse.CalcitePlanner: Get metadata for subqueries 16/05/18 14:40:26 INFO parse.CalcitePlanner: Get metadata for destination tables 16/05/18 14:40:26 INFO metastore.HiveMetaStore: 3: get_table : db=default tbl=keyvalue 16/05/18 14:40:26 INFO HiveMetaStore.audit: ugi=jmill383 ip=unknown-ip-addr cmd=get_table : db=default tbl=keyvalue 16/05/18 14:40:26 INFO parse.CalcitePlanner: Completed getting MetaData in Semantic Analysis 16/05/18 14:40:26 INFO parse.BaseSemanticAnalyzer: Not invoking CBO because the statement has too few joins 16/05/18 14:40:26 INFO common.FileUtils: Creating directory if it doesn't exist: hdfs://localhost:8025/user/hive/warehouse/keyvalue/.hive-staging_hive_2016-05-18_14-40-26_826_7796779660082577343-1 16/05/18 14:40:26 INFO parse.CalcitePlanner: Set stats collection dir : hdfs://localhost:8025/user/hive/warehouse/keyvalue/.hive-staging_hive_2016-05-18_14-40-26_826_7796779660082577343-1/-ext-10001 16/05/18 14:40:27 INFO ppd.OpProcFactory: Processing for FS(2) 16/05/18 14:40:27 INFO ppd.OpProcFactory: Processing for SEL(1) 16/05/18 14:40:27 INFO ppd.OpProcFactory: Processing for TS(0) 16/05/18 14:40:27 INFO log.PerfLogger: <PERFLOG method=partition-retrieving from=org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner> 16/05/18 14:40:27 INFO log.PerfLogger: </PERFLOG method=partition-retrieving start=1463596827044 end=1463596827044 duration=0 from=org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner> 16/05/18 14:40:27 INFO optimizer.GenMRFileSink1: using CombineHiveInputformat for the merge job 16/05/18 14:40:27 INFO physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 16/05/18 14:40:27 INFO physical.NullScanTaskDispatcher: Found 0 null table scans 16/05/18 14:40:27 INFO physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 16/05/18 14:40:27 INFO physical.NullScanTaskDispatcher: Found 0 null table scans 16/05/18 14:40:27 INFO physical.NullScanTaskDispatcher: Looking for table scans where optimization is applicable 16/05/18 14:40:27 INFO physical.NullScanTaskDispatcher: Found 0 null table scans 16/05/18 14:40:27 INFO parse.CalcitePlanner: Completed plan generation 16/05/18 14:40:27 INFO ql.Driver: Semantic Analysis Completed 16/05/18 14:40:27 INFO log.PerfLogger: </PERFLOG method=semanticAnalyze start=1463596826833 end=1463596827060 duration=227 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:27 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:key, type:string, comment:null), FieldSchema(name:value, type:string, comment:null)], properties:null) 16/05/18 14:40:27 INFO log.PerfLogger: </PERFLOG method=compile start=1463596826825 end=1463596827060 duration=235 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:27 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager 16/05/18 14:40:27 INFO log.PerfLogger: <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:27 INFO ql.Driver: Starting command(queryId=jmill383_20160518144026_b7cbea99-d34d-41da-8582-3be591e5d282): insert overwrite table keyvalue select 'Hello' as key, 'hive!' as value from dual Query ID = jmill383_20160518144026_b7cbea99-d34d-41da-8582-3be591e5d282 16/05/18 14:40:27 INFO ql.Driver: Query ID = jmill383_20160518144026_b7cbea99-d34d-41da-8582-3be591e5d282 Total jobs = 3 16/05/18 14:40:27 INFO ql.Driver: Total jobs = 3 16/05/18 14:40:27 INFO log.PerfLogger: </PERFLOG method=TimeToSubmit start=1463596826825 end=1463596827061 duration=236 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:27 INFO log.PerfLogger: <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:27 INFO log.PerfLogger: <PERFLOG method=task.MAPRED.Stage-1 from=org.apache.hadoop.hive.ql.Driver> Launching Job 1 out of 3 16/05/18 14:40:27 INFO ql.Driver: Launching Job 1 out of 3 16/05/18 14:40:27 INFO ql.Driver: Starting task [Stage-1:MAPRED] in serial mode Number of reduce tasks is set to 0 since there's no reduce operator 16/05/18 14:40:27 INFO exec.Task: Number of reduce tasks is set to 0 since there's no reduce operator 16/05/18 14:40:27 INFO ql.Context: New scratch dir is hdfs://localhost:8025/tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1 16/05/18 14:40:27 INFO mr.ExecDriver: Using org.apache.hadoop.hive.ql.io.CombineHiveInputFormat 16/05/18 14:40:27 INFO exec.Utilities: Processing alias dual 16/05/18 14:40:27 INFO exec.Utilities: Adding input file hdfs://localhost:8025/user/hive/warehouse/dual 16/05/18 14:40:27 INFO exec.Utilities: Content Summary not cached for hdfs://localhost:8025/user/hive/warehouse/dual 16/05/18 14:40:27 INFO ql.Context: New scratch dir is hdfs://localhost:8025/tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1 16/05/18 14:40:27 INFO log.PerfLogger: <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities> 16/05/18 14:40:27 INFO exec.Utilities: Serializing MapWork via kryo 16/05/18 14:40:27 INFO log.PerfLogger: </PERFLOG method=serializePlan start=1463596827125 end=1463596827182 duration=57 from=org.apache.hadoop.hive.ql.exec.Utilities> 16/05/18 14:40:27 INFO Configuration.deprecation: mapred.submit.replication is deprecated. Instead, use mapreduce.client.submit.file.replication 16/05/18 14:40:27 ERROR mr.ExecDriver: yarn 16/05/18 14:40:27 INFO client.RMProxy: Connecting to ResourceManager at / 0.0.0.0:8032 16/05/18 14:40:27 INFO fs.FSStatsPublisher: created : hdfs://localhost:8025/user/hive/warehouse/keyvalue/.hive-staging_hive_2016-05-18_14-40-26_826_7796779660082577343-1/-ext-10001 16/05/18 14:40:27 INFO client.RMProxy: Connecting to ResourceManager at / 0.0.0.0:8032 16/05/18 14:40:27 INFO exec.Utilities: PLAN PATH = hdfs://localhost:8025/tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1/-mr-10004/2243cc23-ce83-4a00-87ba-d1970db29def/map.xml 16/05/18 14:40:27 INFO exec.Utilities: PLAN PATH = hdfs://localhost:8025/tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1/-mr-10004/2243cc23-ce83-4a00-87ba-d1970db29def/reduce.xml 16/05/18 14:40:27 INFO exec.Utilities: ***************non-local mode*************** 16/05/18 14:40:27 INFO exec.Utilities: local path = hdfs://localhost:8025/tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1/-mr-10004/2243cc23-ce83-4a00-87ba-d1970db29def/reduce.xml 16/05/18 14:40:27 INFO exec.Utilities: Open file to read in plan: hdfs://localhost:8025/tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1/-mr-10004/2243cc23-ce83-4a00-87ba-d1970db29def/reduce.xml 16/05/18 14:40:27 INFO exec.Utilities: File not found: File does not exist: /tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1/-mr-10004/2243cc23-ce83-4a00-87ba-d1970db29def/reduce.xml at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66) at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1891) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1832) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1812) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1784) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:542) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:362) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033) 16/05/18 14:40:27 INFO exec.Utilities: No plan file found: hdfs://localhost:8025/tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1/-mr-10004/2243cc23-ce83-4a00-87ba-d1970db29def/reduce.xml 16/05/18 14:40:27 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 16/05/18 14:40:27 INFO log.PerfLogger: <PERFLOG method=getSplits from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat> 16/05/18 14:40:27 INFO exec.Utilities: PLAN PATH = hdfs://localhost:8025/tmp/hive/jmill383/b51c81d8-23b9-49ea-b012-92f81bc1c0ce/hive_2016-05-18_14-40-26_826_7796779660082577343-1/-mr-10004/2243cc23-ce83-4a00-87ba-d1970db29def/map.xml 16/05/18 14:40:27 INFO io.CombineHiveInputFormat: Total number of paths: 1, launching 1 threads to check non-combinable ones. 16/05/18 14:40:27 INFO io.CombineHiveInputFormat: CombineHiveInputSplit creating pool for hdfs://localhost:8025/user/hive/warehouse/dual; using filter path hdfs://localhost:8025/user/hive/warehouse/dual 16/05/18 14:40:27 INFO input.FileInputFormat: Total input paths to process : 1 16/05/18 14:40:27 INFO input.CombineFileInputFormat: DEBUG: Terminated node allocation with : CompletedNodes: 1, size left: 0 16/05/18 14:40:27 INFO io.CombineHiveInputFormat: number of splits 1 16/05/18 14:40:27 INFO io.CombineHiveInputFormat: Number of all splits 1 16/05/18 14:40:27 INFO log.PerfLogger: </PERFLOG method=getSplits start=1463596827579 end=1463596827605 duration=26 from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat> 16/05/18 14:40:27 INFO mapreduce.JobSubmitter: number of splits:1 16/05/18 14:40:27 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1463594979064_0004 16/05/18 14:40:27 INFO impl.YarnClientImpl: Submitted application application_1463594979064_0004 16/05/18 14:40:27 INFO mapreduce.Job: The url to track the job: http://starchild:8088/proxy/application_1463594979064_0004/ Starting Job = job_1463594979064_0004, Tracking URL = http://starchild:8088/proxy/application_1463594979064_0004/ 16/05/18 14:40:27 INFO exec.Task: Starting Job = job_1463594979064_0004, Tracking URL = http://starchild:8088/proxy/application_1463594979064_0004/ Kill Command = /opt/hadoop/bin/hadoop job -kill job_1463594979064_0004 16/05/18 14:40:27 INFO exec.Task: Kill Command = /opt/hadoop/bin/hadoop job -kill job_1463594979064_0004 Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0 16/05/18 14:40:31 INFO exec.Task: Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0 16/05/18 14:40:31 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2016-05-18 14:40:31,962 Stage-1 map = 0%, reduce = 0% 16/05/18 14:40:32 INFO exec.Task: 2016-05-18 14:40:31,962 Stage-1 map = 0%, reduce = 0% 16/05/18 14:40:32 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead Ended Job = job_1463594979064_0004 with errors 16/05/18 14:40:32 ERROR exec.Task: Ended Job = job_1463594979064_0004 with errors Error during job, obtaining debugging information... 16/05/18 14:40:32 ERROR exec.Task: Error during job, obtaining debugging information... 16/05/18 14:40:32 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address Job Tracking URL: http://starchild:8088/cluster/app/application_1463594979064_0004 16/05/18 14:40:32 ERROR exec.Task: Job Tracking URL: http://starchild:8088/cluster/app/application_1463594979064_0004 16/05/18 14:40:32 INFO impl.YarnClientImpl: Killed application application_1463594979064_0004 FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask 16/05/18 14:40:32 ERROR ql.Driver: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask 16/05/18 14:40:32 INFO log.PerfLogger: </PERFLOG method=Driver.execute start=1463596827060 end=1463596832133 duration=5073 from=org.apache.hadoop.hive.ql.Driver> MapReduce Jobs Launched: 16/05/18 14:40:32 INFO ql.Driver: MapReduce Jobs Launched: 16/05/18 14:40:32 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL 16/05/18 14:40:32 INFO ql.Driver: Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 0 msec 16/05/18 14:40:32 INFO ql.Driver: Total MapReduce CPU Time Spent: 0 msec 16/05/18 14:40:32 INFO log.PerfLogger: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:32 INFO log.PerfLogger: </PERFLOG method=releaseLocks start=1463596832136 end=1463596832136 duration=0 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:32 INFO log.PerfLogger: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:32 INFO log.PerfLogger: </PERFLOG method=releaseLocks start=1463596832139 end=1463596832139 duration=0 from=org.apache.hadoop.hive.ql.Driver> 16/05/18 14:40:32 WARN cascade.Cascade: [uppercase kv -> kv2 +l...] flow failed: select data from dual into keyvalue cascading.CascadingException: hive error 'FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask' while running query insert overwrite table keyvalue select 'Hello' as key, 'hive!' as value from dual at cascading.flow.hive.HiveQueryRunner.run(HiveQueryRunner.java:131) at cascading.flow.hive.HiveQueryRunner.call(HiveQueryRunner.java:167) at cascading.flow.hive.HiveQueryRunner.call(HiveQueryRunner.java:41) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 16/05/18 14:40:32 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] stopping all flows 16/05/18 14:40:32 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] stopping flow: uppercase kv -> kv2 16/05/18 14:40:32 INFO flow.Flow: [uppercase kv -> kv2 ] stopping all jobs 16/05/18 14:40:32 INFO flow.Flow: [uppercase kv -> kv2 ] stopping: (1/1) .../hive/warehouse/keyvalue2 16/05/18 14:40:32 INFO flow.Flow: [uppercase kv -> kv2 ] stopped all jobs 16/05/18 14:40:32 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] stopping flow: select data from dual into keyvalue 16/05/18 14:40:32 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] stopping flow: load data into dual 16/05/18 14:40:32 INFO cascade.Cascade: [uppercase kv -> kv2 +l...] stopped all flows Exception in thread "main" cascading.cascade.CascadeException: flow failed: select data from dual into keyvalue at cascading.cascade.BaseCascade$CascadeJob.call(BaseCascade.java:963) at cascading.cascade.BaseCascade$CascadeJob.call(BaseCascade.java:900) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: cascading.CascadingException: hive error 'FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask' while running query insert overwrite table keyvalue select 'Hello' as key, 'hive!' as value from dual at cascading.flow.hive.HiveQueryRunner.run(HiveQueryRunner.java:131) at cascading.flow.hive.HiveQueryRunner.call(HiveQueryRunner.java:167) at cascading.flow.hive.HiveQueryRunner.call(HiveQueryRunner.java:41) ... 4 more [jmill383@starchild demo]$ On Wed, May 18, 2016 at 1:48 PM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Hi John, > > can you please a new thread for your problem so we can deal with > separately. > > thanks > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > > On 18 May 2016 at 15:11, JOHN MILLER <jmill...@gmail.com> wrote: > >> Greetings Mitch >> >> I have an issue with running mapreduce in hive I am getting a FAILED: >> Execution Error, return code 2 from >> org.apache.hadoop.hive.ql.exec.mr.MapRedTask >> >> error while attemtiing to execute SELECT DISTINCT(fieldname) FROM TABLE >> x or SELECT COUNT(*) FROM TABLE x;; Trying to run cascading-hive gives >> me the same problem as well >> >> Please advise if u have come across this type of problem or generated >> some ideas as to resolve this problema >> >> On Wed, May 18, 2016 at 9:53 AM, Mich Talebzadeh < >> mich.talebza...@gmail.com> wrote: >> >>> Hi Kuldeep, >>> >>> Have you installed hive on any of these nodes. >>> >>> Hive is basically an API. You will also need to install sqoop as well if >>> you are going to import data from other RDBMss like Oracle, Sybase etc. >>> >>> Hive has a very small footprint so my suggestion is to install it on all >>> your boxes and permission granted to Haddop user say hduser. >>> >>> Hive will require a metadata in a database of your choice. default is >>> derby which I don't use. try to use a reasonable database. ours is on >>> Oracle >>> >>> Now under directory $HIVE_HOME/conf/hive-site.xml you can set up info >>> about Hadoop and your metastore etc. You also need to set up environment >>> variables for both Hadoop and hive in your start up script like .profile >>> .kshrc etc >>> >>> Have a look anyway. >>> >>> HTH >>> >>> Dr Mich Talebzadeh >>> >>> >>> >>> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> >>> http://talebzadehmich.wordpress.com >>> >>> >>> >>> On 18 May 2016 at 13:49, Kuldeep Chitrakar < >>> kuldeep.chitra...@synechron.com> wrote: >>> >>>> I have a very basic question regarding Hadoop & Hive setup. I have 7 >>>> Machines say M1,M2,M3,M4,M5,M6,M7 >>>> >>>> >>>> >>>> Hadoop Cluster Setup: >>>> >>>> >>>> >>>> Namenode: M1 >>>> >>>> Seondary Namenode: M2 >>>> >>>> Datanodes: M3,M4,M5 >>>> >>>> >>>> >>>> >>>> >>>> Now question is: >>>> >>>> >>>> >>>> Where do I need to install Hive. >>>> >>>> 1. Should I install Hiverserver on M6 >>>> >>>> a. if yes does that machine needs core Hadoop JAR’s installed? >>>> >>>> b. How this Hive server knows where Hadoop cluster is. What >>>> configurations needs to be done? >>>> >>>> c. How can we restrict this machine to be only hive server and >>>> not datanode of Hadoop cluster? >>>> >>>> >>>> >>>> 2. Where do we install Hive CLI >>>> >>>> a. If I want to hive M7 as Hive CLI, then what needs to be >>>> installed on this machine. >>>> >>>> b. Any required configurations. >>>> >>>> >>>> >>>> Thanks, >>>> >>>> Kuldeep >>>> >>>> >>>> >>> >>> >> >
