Hi Kumar and Andrea, this is a known change in Flink behaviour from 1.4 to 1.5 (after FLIP-6). There is an issue to track progress on more fine-grained task distribution [1].
Best, Andrey [1] https://issues.apache.org/jira/browse/FLINK-11815 On Mon, Mar 18, 2019 at 1:28 PM Kumar Bolar, Harshith <hk...@arity.com> wrote: > We recently upgraded to 1.7.2. Previously in 1.4.2 the jobs were getting > deployed in a round robin manner. First job goes to TM1, second job goes to > TM2 and so on. But after upgrading to 1.7.2, all jobs are going to the same > task manager until the slots there run out. Did something change in 1.7? > > > > Thanks, > > Harshith > > > > *From: *Andrea Spina <andrea.sp...@radicalbit.io> > *Date: *Monday, 18 March 2019 at 5:54 PM > *To: *Harshith Kumar Bolar <hk...@arity.com> > *Cc: *user <user@flink.apache.org> > *Subject: *[External] Re: Flink 1.7.2: All jobs are getting deployed on > the same task manager > > > > Hi everybody. We're currently experimenting the same characteristic on > flink-1.6.2. > > I've been reading that Flink treats all the slot as equals, it doesn't > even know where these slots reside > https://stackoverflow.com/questions/54980104/uneven-assignment-of-tasks-to-workers-in-flink > <https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_questions_54980104_uneven-2Dassignment-2Dof-2Dtasks-2Dto-2Dworkers-2Din-2Dflink&d=DwMFaQ&c=gtIjdLs6LnStUpy9cTOW9w&r=61bFb6zUNKZxlAQDRo_jKA&m=x7Ae5P9d7Ni3EkdbM3aw6gaRzqYcCoFlSAmXCz-8n-A&s=mLWGWKLiWiUyx1n_16tJQt3E3iAbaoi4GMH40WordpA&e=>. > > So it should not be an issue; thus, the fact that it runs all the slots of > a machine before moving to a new one should be just a rough coincidence. > > Given that, I'm pretty sure that I've never been recording this feature > using previous majors (I recall flink-1.3 for sure). > Moreover, this is damaging because you can get resources exhausted (e.g. > memory, disk). > > Hope we might find a solution on this. > Sincerely, > > Andrea > > > > Il giorno lun 18 mar 2019 alle ore 11:53 Kumar Bolar, Harshith < > hk...@arity.com> ha scritto: > > Hi all, > > > > We're running a Flink on a five node standalone cluster with three task > manager (TM1, TM2, TM3) and two job managers. > > > > Whenever I submit a new job, the job gets deployed on only TM3. When the > number of slots in TM3 get exhausted, the jobs start getting deployed on > TM2 and so on. How do I ensure that the jobs get distributed evenly across > all 3 task managers? > > > > Thanks, > > Harshith > > > > > > > -- > > *Andrea Spina* > Software Engineer @ Radicalbit Srl > Via Borsieri 41, 20159, Milano - IT >
