Hi Jagat,
I think your issue here are not the JVM options. You are missing shell
environment variables during the container launch. Adding those to the
user's .bashrc or .profile should fix the problem.
Best regards,
Max
On Mon, Nov 23, 2015 at 10:14 PM, Jagat Singh <jagatsi...@gmail.com> wrote:
> Hello Robert,
>
> Added following
>
> env.java.opts: "-Dstack.name=phd -Dstack.version=3.0.0.0-249"
>
> Same Error
>
> Is there any config which allows to pass special java opts to actual yarn
> containers?
>
> Thanks,
>
> Jagat Singh
>
>
>
>
>
> On Mon, Nov 23, 2015 at 9:21 PM, Robert Metzger <rmetz...@apache.org> wrote:
>>
>> Hi,
>>
>> In Flink the configuration parameter for passing custom JVM options is
>> "env.java.opts". I would recommend to put it into the conf/flink-config.yaml
>> like this:
>>
>> env.java.opts: "-Dhdp.version=2.3.0.0-2557 -Dhdp.version=2.3.0.0-2557"
>>
>> Please let me know if this works.
>> Maybe you are the first user running Flink on Pivotal HDP and there are
>> some things different to other Hadoop distributions.
>>
>> Regards,
>> Robert
>>
>>
>>
>>
>> On Mon, Nov 23, 2015 at 1:15 AM, Jagat Singh <jagatsi...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I am running example Flink program (Pivotal HDP)
>>>
>>> ./bin/flink run -m yarn-cluster -yn 2 ./examples/WordCount.jar
>>>
>>> I am getting error below.
>>>
>>> How to pass the stack.name and stack.version to the flink program.
>>>
>>> This is similar to what we give to Spark as
>>>
>>> hdp.version.
>>>
>>> Example
>>>
>>> spark.driver.extraJavaOptions -Dhdp.version=2.3.0.0-2557
>>> spark.yarn.am.extraJavaOptions -Dhdp.version=2.3.0.0-2557
>>>
>>> Thanks
>>>
>>> Exception message:
>>> /grid/0/hadoop/yarn/local/usercache/d760770/appcache/application_1447977375774_17024/container_e34_1447977375774_17024_01_000001/launch_container.sh:
>>> line 26:
>>> $PWD/*:$HADOOP_CONF_DIR:/usr/${stack.name}/current/hadoop-client/*:/usr/${stack.name}/current/hadoop-client/lib/*:/usr/${stack.name}/current/hadoop-hdfs-client/*:/usr/${stack.name}/current/hadoop-hdfs-client/lib/*:/usr/${stack.name}/current/hadoop-yarn-client/*:/usr/${stack.name}/current/hadoop-yarn-client/lib/*:
>>> bad substitution
>>>
>>> Stack trace: ExitCodeException exitCode=1:
>>> /grid/0/hadoop/yarn/local/usercache/d760770/appcache/application_1447977375774_17024/container_e34_1447977375774_17024_01_000001/launch_container.sh:
>>> line 26:
>>> $PWD/*:$HADOOP_CONF_DIR:/usr/${stack.name}/current/hadoop-client/*:/usr/${stack.name}/current/hadoop-client/lib/*:/usr/${stack.name}/current/hadoop-hdfs-client/*:/usr/${stack.name}/current/hadoop-hdfs-client/lib/*:/usr/${stack.name}/current/hadoop-yarn-client/*:/usr/${stack.name}/current/hadoop-yarn-client/lib/*:
>>> bad substitution
>>>
>>>
>>
>
