Re: Problem with Amazon S3

2015-04-01 Thread Ufuk Celebi
Hey Pietro, I've debugged this locally and I can get a connection to a S3 bucket with the following format: s3://.s3.amazonaws.com/ Depending on the region of your S3 bucket, you have to use a different endpoint (http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region ). So for exampl

Re: GC on taskmanagers

2015-04-01 Thread Maximilian Michels
Hi Emmanuel, If a job fails due to an Exception being thrown, the job is canceled. The task manager remains intact and further jobs can be submitted. An OutOfMemoryError, as the name implies, is not an Exception but an Error. In general, this error is thrown when you make excessive use of Object

Re: Problem with Amazon S3

2015-04-01 Thread pietro
Hi Stephan, for sure I can: this is how I try to read from Flink: env.readFile(new DefaultReader(), "s3://genomic/flink/input/meta/1.txt" ).map(parser(_))} I use the same format I was used to use in Pig. Thanks, -- View this message in context: http://apache-flink-incubator-user-mailing-l

Re: Problem with Amazon S3

2015-04-01 Thread Stephan Ewen
It looks like the S3 URL is in an unexpected format. It tries to use the bucketname as the hostname. Can you tell us the S3 URL (without user / password), so we can take a look? Greetings, Stephan On Tue, Mar 31, 2015 at 12:52 PM, pietro wrote: > Thank you Ufuk! That helped a lot. > > But I ha