Re: Subtask much slower than the others when creating checkpoints

2019-01-15 Thread Till Rohrmann
> The subtask that seems causing problems seems to be a CoProcessFunction. > I am going to debug Flink but since I'm relatively new to it, it might > take a while so any help will be appreciated. > > Pasquale > > > From: Till Rohrmann > Sent: 08 January 2019 17:3

RE: Subtask much slower than the others when creating checkpoints

2019-01-15 Thread Pasquale Vazzana
mann mailto:trohrm...@apache.org>> Sent: 08 January 2019 17:35 To: Bruno Aranda mailto:bara...@apache.org>> Cc: user mailto:user@flink.apache.org>> Subject: Re: Subtask much slower than the others when creating checkpoints Hi Bruno, there are multiple reasons wh= one of the sub

Re: Subtask much slower than the others when creating checkpoints

2019-01-15 Thread Bruno Aranda
problem. >>> I've been wondering whether some Kafka partitions where empty but there >>> is not much data skew and the keyBy uses the same key strategy as the Kafka >>> partitions, I've tried to use murmur2 for hashing but it didn't help either. >>&

Re: Subtask much slower than the others when creating checkpoints

2019-01-15 Thread Stefan Richter
g problems seems to be a CoProcessFunction. > I am going to debug Flink but since I'm relatively new to it, it might take a > while so any help will be appreciated. > > Pasquale > > > From: Till Rohrmann mailto:trohrm...@apache.org>> > Sent: 08 January 2019 17

Re: Subtask much slower than the others when creating checkpoints

2019-01-15 Thread Bruno Aranda
since I'm relatively new to it, it might >> take a while so any help will be appreciated. >> >> Pasquale >> >> >> From: Till Rohrmann >> Sent: 08 January 2019 17:35 >> To: Bruno Aranda >> Cc: user >> Subject: Re: Subtask muc

Re: Subtask much slower than the others when creating checkpoints

2019-01-15 Thread Till Rohrmann
e > > > From: Till Rohrmann > Sent: 08 January 2019 17:35 > To: Bruno Aranda > Cc: user > Subject: Re: Subtask much slower than the others when creating checkpoints > > Hi Bruno, > > there are multiple reasons wh= one of the subtasks can take longer for >

RE: Subtask much slower than the others when creating checkpoints

2019-01-14 Thread Pasquale Vazzana
to it, it might take a while so any help will be appreciated. Pasquale From: Till Rohrmann Sent: 08 January 2019 17:35 To: Bruno Aranda Cc: user Subject: Re: Subtask much slower than the others when creating checkpoints Hi Bruno, there are multiple reasons wh= one of the subtasks can take

Re: Subtask much slower than the others when creating checkpoints

2019-01-08 Thread Till Rohrmann
Hi Bruno, there are multiple reasons why one of the subtasks can take longer for checkpointing. It looks as if there is not much data skew since the state sizes are relatively equal. It also looks as if the individual tasks all start at the same time with the checkpointing which indicates that the