Hi,

did you observe the problem only under YARN 2.4.0? IIRC this version of YARN 
has some problems that can also lead to issues with Flink’s HA mode, and I 
would encourage you to upgrade YARN to 2.5 or higher. On a different note, 
there have been several improvements that we will release in Flink 1.1.4, not 
entirely sure if this is a known problem covered by the upcoming bugfix 
release. I will add Till to the discussion who worked a lot in this direction.

Best,
Stefan

> Am 07.12.2016 um 09:19 schrieb Michael Pisula <[email protected]>:
> 
> Hi Guys,
> 
> We are having a slight issue using Flink 1.1.3 (we also observed the
> problem with 1.0.2) in Yarn 2.4.0. Whenever a TaskManager restarts, YARN
> seems to reserve memory during the TaskManager restart, and not free the
> memory again. We are using a CapacityScheduler with 2 queues, where the
> queue in which our Flink Yarn Session runs has a guaranteed capacity of
> 75%. What we are seeing, is that the amount of reserved memory is
> exactly the amount of memory available in the queue after the
> TaskManager is crashed.
> 
> On our test system, further TaskManager restarts have been able to get
> rid of the TaskManager again. When trying to replicate this on our
> production system I was not successful, one difference being, that I
> killed a TaskManager with no used slots in prod, while on the test
> system jobs were restarted.
> 
> Nothing enlightening in the logs, unfortunately.
> 
> Is this something that anyone has experienced so far?
> 
> Cheers,
> 
> Michael
> 
> 
> -- 
> Michael Pisula * [email protected] * +49-174-3180084
> TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
> Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
> Sitz: Unterföhring * Amtsgericht München * HRB 135082
> 
> 

Reply via email to