Another way to do this would be to set a property in the Hadoop config itself.
In the job launcher you would have something like:
JobConf conf = ...
conf.setProperty("foo", "test");
Then you can read the property in your map or reduce task.
Tom
On Thu, Apr 30, 2009 at 3:25 PM, Aaron Kimball <[email protected]> wrote:
> So you want a different "-Dfoo=test" on each node? It's probably grabbing
> the setting from the node where the job was submitted, and this overrides
> the settings on each task node.
>
> Try adding <final>true</final> to the <property> block on the tasktrackers,
> then restart Hadoop and try again. This will prevent the job from overriding
> the setting.
>
> - Aaron
>
> On Thu, Apr 30, 2009 at 9:25 AM, Marc Limotte <[email protected]> wrote:
>
>> I'm trying to set a System Property in the Hadoop config, so my jobs will
>> know which cluster they are running on. I think I should be able to do this
>> with -Dname=value in "mapred.child.java.opts" (example below), but the
>> setting is ignored.
>> In hadoop-site.xml I have:
>> <property>
>> <name>mapred.child.java.opts</name>
>> <value>-Xmx200m -Dfoo=test</value>
>> </property>
>> But the job conf through the web server indicates:
>> mapred.child.java.opts -Xmx1024M -Duser.timezone=UTC
>>
>> I'm using Hadoop-0.17.2.1.
>> Any tips on why my setting is not picked up?
>>
>> Marc
>>
>> ________________________________
>> PRIVATE AND CONFIDENTIAL - NOTICE TO RECIPIENT: THIS E-MAIL IS MEANT FOR
>> ONLY THE INTENDED RECIPIENT OF THE TRANSMISSION, AND MAY BE A COMMUNICATION
>> PRIVILEGE BY LAW. IF YOU RECEIVED THIS E-MAIL IN ERROR, ANY REVIEW, USE,
>> DISSEMINATION, DISTRIBUTION, OR COPYING OF THIS EMAIL IS STRICTLY
>> PROHIBITED. PLEASE NOTIFY US IMMEDIATELY OF THE ERROR BY RETURN E-MAIL AND
>> PLEASE DELETE THIS MESSAGE FROM YOUR SYSTEM.
>>
>
