We recently upgraded to 1.7.2. Previously in 1.4.2 the jobs were getting deployed in a round robin manner. First job goes to TM1, second job goes to TM2 and so on. But after upgrading to 1.7.2, all jobs are going to the same task manager until the slots there run out. Did something change in 1.7?
Thanks, Harshith From: Andrea Spina <andrea.sp...@radicalbit.io> Date: Monday, 18 March 2019 at 5:54 PM To: Harshith Kumar Bolar <hk...@arity.com> Cc: user <user@flink.apache.org> Subject: [External] Re: Flink 1.7.2: All jobs are getting deployed on the same task manager Hi everybody. We're currently experimenting the same characteristic on flink-1.6.2. I've been reading that Flink treats all the slot as equals, it doesn't even know where these slots reside https://stackoverflow.com/questions/54980104/uneven-assignment-of-tasks-to-workers-in-flink<https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_questions_54980104_uneven-2Dassignment-2Dof-2Dtasks-2Dto-2Dworkers-2Din-2Dflink&d=DwMFaQ&c=gtIjdLs6LnStUpy9cTOW9w&r=61bFb6zUNKZxlAQDRo_jKA&m=x7Ae5P9d7Ni3EkdbM3aw6gaRzqYcCoFlSAmXCz-8n-A&s=mLWGWKLiWiUyx1n_16tJQt3E3iAbaoi4GMH40WordpA&e=>. So it should not be an issue; thus, the fact that it runs all the slots of a machine before moving to a new one should be just a rough coincidence. Given that, I'm pretty sure that I've never been recording this feature using previous majors (I recall flink-1.3 for sure). Moreover, this is damaging because you can get resources exhausted (e.g. memory, disk). Hope we might find a solution on this. Sincerely, Andrea Il giorno lun 18 mar 2019 alle ore 11:53 Kumar Bolar, Harshith <hk...@arity.com<mailto:hk...@arity.com>> ha scritto: Hi all, We're running a Flink on a five node standalone cluster with three task manager (TM1, TM2, TM3) and two job managers. Whenever I submit a new job, the job gets deployed on only TM3. When the number of slots in TM3 get exhausted, the jobs start getting deployed on TM2 and so on. How do I ensure that the jobs get distributed evenly across all 3 task managers? Thanks, Harshith -- Andrea Spina Software Engineer @ Radicalbit Srl Via Borsieri 41, 20159, Milano - IT
