Passing Java system properties to YARN containers of Samza Job

Hi, Is there any convenient way to pass custom Java system properties to Samza jobs? Specifically, since the system properties I wish to pass are aws.accessKeyId and aws.secretKey for AWS credentials ( http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/credentials.html), writing them

Re: Passing Java system properties to YARN containers of Samza Job

Gordon, If your tasks run on EC2, you should use IAM roles. https://www.youtube.com/watch?v=XuRM4Id6uDY Kind regards,
 Radek Gruchalski 
ra...@gruchalski.com (mailto:ra...@gruchalski.com)
 (mailto:ra...@gruchalski.com) de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruc

Re: Review Request 41071: SAMZA-843: fix heap usage increase caused by container timer change

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41071/#review109387 --- Ship it! Ship It! - Yi Pan (Data Infrastructure) On Dec. 8, 201

Re: Review Request 41071: SAMZA-843: fix heap usage increase caused by container timer change

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41071/#review109415 --- samza-api/src/main/java/org/apache/samza/metrics/SlidingTimeWindow

Re: Review Request 41071: SAMZA-843: fix heap usage increase caused by container timer change

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41071/#review109423 --- Ship it! We should consider more permanent fix later - may be usin

Review Request 41106: SAMZA-833: ProcessJob mishandling containers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41106/ --- Review request for samza. Repository: samza Description --- throws runti

Re: Review Request 41068: SAMZA-813: Add Seek functionality to KeyValueStoreIterator

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41068/#review109417 --- @Amit, thanks for putting up this patch. I have two high-level comm

Re: Review Request 41106: SAMZA-833: ProcessJob mishandling containers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41106/#review109428 --- Thanks, Tao! One issue in the comment. samza-core/src/main/scala/

Re: [VOTE] [CANCEL] Apache Samza 0.10.0 RC1

Pulling in SAMZA-834 due to perf degradation detected recently. On Mon, Dec 7, 2015 at 3:24 PM, Yi Pan wrote: > Hey all, > > > This is a call for a vote on a release of Apache Samza 0.10.0. Thanks to > everyone who has contributed to this release. We are very glad to see some > new contributors

[VOTE] Samza 0.10.0 Release Candidate 2

Hey all, This is a call for a vote on a release of Apache Samza 0.10.0. Thanks to everyone who has contributed to this release. We are very glad to see some new contributors in this release. The release candidate can be downloaded from here: http://home.apache.org/~nickpan47/samza-0.10.0-rc2/

Re: [VOTE] [CANCEL] Apache Samza 0.10.0 RC1

My +1, on the release. This has many great features! On Tue, Dec 8, 2015 at 3:26 PM, Yi Pan wrote: > Pulling in SAMZA-834 due to perf degradation detected recently. > > On Mon, Dec 7, 2015 at 3:24 PM, Yi Pan wrote: > > > Hey all, > > > > > > This is a call for a vote on a release of Apache Samz

Re: [VOTE] Samza 0.10.0 Release Candidate 2

+1 Approve from my side. On Tue, Dec 8, 2015 at 3:38 PM, Yi Pan wrote: > Hey all, > > > This is a call for a vote on a release of Apache Samza 0.10.0. Thanks to > everyone who has contributed to this release. We are very glad to see some > new contributors in this release. > > > The release cand

Re: Passing Java system properties to YARN containers of Samza Job

Hi Rad, Our Samza jobs run on on-premiere clusters, on top of YARN and Kafka. So IAM roles won't be an option either. BR, On 9 December 2015 at 04:09, Rad Gruchalski wrote: > Gordon, > > If your tasks run on EC2, you should use IAM roles. > https://www.youtube.com/watch?v=XuRM4Id6uDY > > > > >

Re: Passing Java system properties to YARN containers of Samza Job

Hi, Gordon, Try to use --config X.Y.Z=value from the command line to run run-job.sh On Tue, Dec 8, 2015 at 5:57 PM, Gordon Tai wrote: > Hi Rad, > > Our Samza jobs run on on-premiere clusters, on top of YARN and Kafka. So > IAM roles won't be an option either. > > BR, > > On 9 December 2015 at 0

Re: Review Request 41106: SAMZA-833: ProcessJob mishandling containers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41106/ --- (Updated Dec. 9, 2015, 6:41 a.m.) Review request for samza. Repository: samza

Re: Review Request 41106: SAMZA-833: ProcessJob mishandling containers

> On Dec. 8, 2015, 11:20 p.m., Yi Pan (Data Infrastructure) wrote: > > samza-core/src/main/scala/org/apache/samza/job/local/ProcessJobFactory.scala, > > line 54 > > > > > > It would be better to use JobConfig.getCont