So I finally got a couple of test scripts running on my cluster to take
a sample data file, load it, do a little processing, store it, load it,
do a little more processing, and dump the results.
Once these were working, I set to parsing and storing some real data,
but when got an "Unable to create input slice" error when trying to load
this data back out again. This happened with each of:
foo = LOAD '/path/to/file/{item,list,glob}/*/subdir' USING
com.twitter.elephantbird.pig.load.LzoTokenizedLoader(',') AS (schema:...);
foo = LOAD '/path/to/file/item/*/subdir' USING
com.twitter.elephantbird.pig.load.LzoTokenizedLoader(',') AS (schema:...);
foo = LOAD '/path/to/file/item/ex/subdir' USING
com.twitter.elephantbird.pig.load.LzoTokenizedLoader(',') AS (schema:...);
and yielded the error (the same each time, except for the name/glob
used):
ERROR 2997: Unable to recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to
create input slice for: hdfs://master.hadoop:9000//path/to/file/item/ex/subdir
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open
iterator for alias foo
at org.apache.pig.PigServer.openIterator(PigServer.java:482)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:539)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75)
at org.apache.pig.Main.main(Main.java:352)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2997:
Unable to recreate exception from backend error:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to
create input slice for: hdfs://master.hadoop:9000/path/to/file/item/ex/subdir
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:176)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:253)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:249)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:781)
at org.apache.pig.PigServer.store(PigServer.java:529)
at org.apache.pig.PigServer.openIterator(PigServer.java:465)
... 6 more
Anyone have any suggestions why this may be happening and how to fix it?
Thanks,
Kris
--
Kris Coward http://unripe.melon.org/
GPG Fingerprint: 2BF3 957D 310A FEEC 4733 830E 21A4 05C7 1FEB 12B3
