Re: samza environment variable on containers

2015-08-03 Thread Yan Fang
Maybe @Eli Reisman can give you some insights, since he is writing the HDFS producer. Because the exception looks like related to the hdfs consumer. Thanks, Fang, Yan yanfang...@gmail.com On Sun, Aug 2, 2015 at 12:41 PM, Chen Song wrote: > Thanks Yan. > > I ran into issues when testing jobs on

Re: samza environment variable on containers

2015-08-02 Thread Chen Song
Thanks Yan. I ran into issues when testing jobs on kerberized cluster. The job reads from HDFS and it worked well before. After testing on kerberized cluster, the Samza container threw exception as below. I am not sure how kerberos has anything to do with this. java.lang.UnsatisfiedLinkError: org

Re: samza environment variable on containers

2015-07-30 Thread Yan Fang
Hi Chen Song, I do not think there is a way in Samza with which you can specify the ENV for Samza container. And currently Samza does not read the LD_LIBRARY_PATH either. Samza only puts the files in lib/*.[jw]ar into the CLASSPATH. Though -Djava.library.path might work, it will cause hadoop e

samza environment variable on containers

2015-07-30 Thread Chen Song
Maybe a dumb question. Is there a way to set an ENV for samza containers? We want to set LD_LIBRARY_PATH to include hadoop native libs. -- Chen Song