Hello PigUsers,
I¹m running a series of Pig scripts from a java program, and I find that
it¹s tough to swap out configurations
For instance,
- when I run a local build I¹d like to run pig in local mode,
- when I run an automated build I¹d like to connect to a QA Hadoop cluster,
- when I deploy to prod I¹d like to connect to a larger Production cluster.
I was under the impression that the PigServer(ExecType, Properties)
consturctor would let me set "fs.default.name" and "mapred.job.tracker" -
which would accomplish exactly what I want - I could externalize those
properties and swap them out depending on the environment. However, it looks
like HExecutionEngine is going to try to pick the XML files up regardless of
what I put into the constructor argument for Properties.
(ref: line 167 of HExecutionEngine)
http://search-hadoop.com/c/Pig:/src/org/apache/pig/backend/hadoop/executione
ngine/HExecutionEngine.java%7C%7ChexecutionEngine
Does anyone know..
1) If there's a way to programmatically set "fs.default.name"/
"mapred.job.tracker" WITHOUT requiring XML files to be in the classpath?
2) What does the properties parameter in PigServer(ExecType, Properties) do?
I could find it anywhere in the javadocs or the source code - I would be
happy to update the javadocs if someone can help me figure out what this is
doing.
Thank you,
- Andrew
