Hi Sebastian, There is no way to return memory from a Flink process except shutting the process down. I think YARN could help in your setup. In a YARN setup, you can flexibly start and stop Flink sessions with different configurations (memory, TMs, slots) or run a single job. When running a single job, Flink will allocate resources and free them after the job is done.
Best, Fabian 2015-12-09 9:46 GMT+01:00 Kruse, Sebastian <sebastian.kr...@hpi.de>: > Hi everyone, > > > I am currently looking into how Flink can coexist and interoperate with > other frameworks in a cluster, such as plain single-machine processes > or Spark. Tachyon seems to be nice solution to exchange data between > them. > > > However, I think it is a problem that Flink's taskmanagers allocate their > managed > memory upfront - in contrast to Spark, as far as I know. If I want a > taskmanager > to yield its main memory, so that another process can use that memory, is > there any other option besides shutting that taskmanager down? Would it be > beneficial to use YARN? > > Thanks for your help! > > > Cheers, > > Sebastian >
