I get the output: rw-r--r-- 2 kris supergroup 172694 2011-02-25 01:59 /path/to/file/item/ex/subdir
-K On Tue, Mar 01, 2011 at 12:46:31PM -0800, Dmitriy Ryaboy wrote: > What happens when you "hadoop fs -lsr" those paths? > > D > > On Sun, Feb 27, 2011 at 7:47 PM, Kris Coward <[email protected]> wrote: > > > > > So I finally got a couple of test scripts running on my cluster to take > > a sample data file, load it, do a little processing, store it, load it, > > do a little more processing, and dump the results. > > > > Once these were working, I set to parsing and storing some real data, > > but when got an "Unable to create input slice" error when trying to load > > this data back out again. This happened with each of: > > > > foo = LOAD '/path/to/file/{item,list,glob}/*/subdir' USING > > com.twitter.elephantbird.pig.load.LzoTokenizedLoader(',') AS (schema:...); > > foo = LOAD '/path/to/file/item/*/subdir' USING > > com.twitter.elephantbird.pig.load.LzoTokenizedLoader(',') AS (schema:...); > > foo = LOAD '/path/to/file/item/ex/subdir' USING > > com.twitter.elephantbird.pig.load.LzoTokenizedLoader(',') AS (schema:...); > > > > and yielded the error (the same each time, except for the name/glob > > used): > > > > ERROR 2997: Unable to recreate exception from backend error: > > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to > > create input slice for: > > hdfs://master.hadoop:9000//path/to/file/item/ex/subdir > > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to > > open iterator for alias foo > > at org.apache.pig.PigServer.openIterator(PigServer.java:482) > > at > > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:539) > > at > > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241) > > at > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) > > at > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144) > > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75) > > at org.apache.pig.Main.main(Main.java:352) > > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR > > 2997: Unable to recreate exception from backend error: > > org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to > > create input slice for: > > hdfs://master.hadoop:9000/path/to/file/item/ex/subdir > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:176) > > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:253) > > at > > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:249) > > at > > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:781) > > at org.apache.pig.PigServer.store(PigServer.java:529) > > at org.apache.pig.PigServer.openIterator(PigServer.java:465) > > ... 6 more > > > > > > Anyone have any suggestions why this may be happening and how to fix it? > > > > Thanks, > > Kris > > > > -- > > Kris Coward http://unripe.melon.org/ > > GPG Fingerprint: 2BF3 957D 310A FEEC 4733 830E 21A4 05C7 1FEB 12B3 > > -- Kris Coward http://unripe.melon.org/ GPG Fingerprint: 2BF3 957D 310A FEEC 4733 830E 21A4 05C7 1FEB 12B3
