Re: Flink in EMR configuration problem

2020-04-01 Thread Piotr Nowojski
Hi, Sorry I missed that. But yes, it looks like you are running two JobManagers :) You can always check the yarn logs for more information what is being executed. Piotrek > On 1 Apr 2020, at 16:44, Antonio Martínez Carratalá > wrote: > > Hi Piotr, > > I don't have 2 task managers, just one

Re: Flink in EMR configuration problem

2020-04-01 Thread Antonio Martínez Carratalá
Hi Piotr, I don't have 2 task managers, just one with 2 slots. That would be ok according to my calculations, but as Craig said I need one more instance for the cluster master. I was guessing the job manager was running in the master and the task manager in the slave, but both job manager and task

Re: Flink in EMR configuration problem

2020-04-01 Thread Piotr Nowojski
Hey, Isn’t explanation of the problem in the logs that you posted? Not enough memory? You have 2 EMR nodes, 8GB memory each, while trying to allocate 2 TaskManagers AND 1 JobManager with 6GB heap size each? Piotrek > On 31 Mar 2020, at 17:01, Antonio Martínez Carratalá > wrote: > > Hi, I'm

Flink in EMR configuration problem

2020-03-31 Thread Antonio Martínez Carratalá
Hi, I'm trying to run a job in a Flink cluster in Amazon EMR from java code but I'm having some problems This is how I create the cluster: StepConfig copyJarStep = new StepConfig() .wit