Re: Deploying Samza Jobs Using S3 and YARN on AWS

Glad that you found the issue Xiaochuan! Should you decide to use the HttpFileSystem, Please set the config for yarn.package.path to point to the HTTP URI of your job's binary. Do let us know should you hit snags down that path! Best, On Sat, Sep 23, 2017 at 5:24 PM XiaoChuan Yu wrote: > I fou

Re: Deploying Samza Jobs Using S3 and YARN on AWS

I found out that it was necessary to include "hadoop-aws" as a part of the package submitted to YARN similar to the instructions for deploying from HDFS . However, due to a dependency conflict on the AWS SDK between ou

Re: Deploying Samza Jobs Using S3 and YARN on AWS

Thank you Xiaochuan for your question! You should ensure that *every machine in your cluster* has the S3 jar file in its YARN class-path. From your error, it looks like the machine you are running on does not have the JAR file corresponding to *S3AFileSystem*. >> Whats the right way to set this u

Deploying Samza Jobs Using S3 and YARN on AWS

Hi, I'm trying to deploy a Samza job using YARN and S3 where I upload the zip package to S3 and point yarn.package.path to it. Does anyone know what kind of set up steps is required for this? What I've tried so far is to get Hello Samza to be run this way in AWS. However I ran into the following