Hi all,
I am experiencing a similar problem but with HDFS as a source instead of
Kafka. I have a streaming pipeline as follows:
1 - read a folder continuousely from HDFS
2 - filter duplicates (using keyby(x->x) and keeping a state per key
indicating whether its is seen)
3 - schedule some future ac
BRILLIANT !!!
Checkpoint times are consistent with 1.1.4...
Thanks for your formidable support !
Best Regards
CVP
On Wed, Jan 4, 2017 at 5:33 PM, Fabian Hueske wrote:
> Hi CVP,
>
> we recently release Flink 1.1.4, i.e., the next bugfix release of the
> 1.1.x series with major robustness impro
Hi CVP,
we recently release Flink 1.1.4, i.e., the next bugfix release of the 1.1.x
series with major robustness improvements [1].
You might want to give 1.1.4 a try as well.
Best, Fabian
[1] http://flink.apache.org/news/2016/12/21/release-1.1.4.html
2017-01-04 16:51 GMT+01:00 Chakravarthy vara
Hi Stephan, All,
I just got a chance to try if 1.1.3 fixes slow check pointing on FS
backend. It seemed to have been fixed. Thanks for the fix.
While testing this, with varying check point intervals, there seem to
be Spikes of slow checkpoints every 30/40 seconds for an interval of 15
s
Thanks for your prompt response Stephan.
I'd wait for Flink 1.1.3 !!!
Best Regards
Varaga
On Tue, Oct 4, 2016 at 5:36 PM, Stephan Ewen wrote:
> The plan to release 1.1.3 is asap ;-)
>
> Waiting for last backported patched to get in, then release testing and
> release.
>
> If you want to te
The plan to release 1.1.3 is asap ;-)
Waiting for last backported patched to get in, then release testing and
release.
If you want to test it today, you would need to manually build the
release-1.1 branch.
Best,
Stephan
On Tue, Oct 4, 2016 at 5:46 PM, Chakravarthy varaga <
chakravarth...@gmail
Hi Gordon,
Do I need to clone and build release-1.1 branch to test this?
I currently use flinlk 1.1.2 runtime. When is the plan to release it
in 1.1.3?
Best Regards
Varaga
On Tue, Oct 4, 2016 at 9:25 AM, Tzu-Li (Gordon) Tai
wrote:
> Hi,
>
> Helping out here: this is the PR for async
Hi,
Helping out here: this is the PR for async Kafka offset committing -
https://github.com/apache/flink/pull/2574.
It has already been merged into the master and release-1.1 branches, so you can
try out the changes now if you’d like.
The change should also be included in the 1.1.3 release, whic
Hi Stephan,
Is the Async kafka offset commit released in 1.3.1?
Varaga
On Wed, Sep 28, 2016 at 9:49 AM, Chakravarthy varaga <
chakravarth...@gmail.com> wrote:
> Hi Stephan,
>
> That should be great. Let me know once the fix is done and the
> snapshot version to use, I'll check and reve
Hi Stephan,
That should be great. Let me know once the fix is done and the
snapshot version to use, I'll check and revert then.
Can you also share the JIRA that tracks the issue?
With regards to offset commit issue, I'm not sure as to how to proceed
here. Probably I'll use your fix
t;>>>>>
>>>>>>>>>> KeyedStream *ks1* = ds1.keyBy("*") ;
>>>>>>>>>> KeyedStream, String> *ks2* =
>>>>>>>>>> ds2.flatMap(split T into k-v pairs).keyBy(0);
>>>>>>>>>>
>>>&g
gt; ble.com/Flink-Checkpoint-runs-slow-for-low-load-stream-tp914
>>>>>> 7p9176.html
>>>>>> To start a new topic under Apache Flink User Mailing List archive.,
>>>>>> email [hidden email]
>>>>>> <http:///user/SendEmail.j
@CVP
Flink stores in checkpoints in your case only the Kafka offsets (few bytes)
and the custom state (e).
Here is an illustration of the checkpoint and what is stored (from the
Flink docs).
https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html
I am quite pu
ents/sec from kafka topic
>>>>>>>> //ks2 is streaming about 1 event every 10 minutes... Precisely
>>>>>>>> when the 1st event is consumed from this stream, checkpoint takes 2
>>>>>>>> minutes
>>>>>>>> stra
Hi Stefan,
Thanks a million for your detailed explanation. I appreciate it.
- The *zookeeper bundled with kafka 0.9.0.1* was used to start
zookeeper. There is only 1 instance (standalone) of zookeeper running on my
localhost (ubuntu 14.04)
- There is only 1 Kafka broker (*version
checkpoint every 10 Secs using a FsStateBackend...
>>>>>>> What I notice is that the checkpoint duration is almost 2 minutes for
>>>>>>> many
>>>>>>> cases, while for the other cases it varies from 100 ms to 1.5 minutes
>>>>>>>
;>>>>
>>>>>> Best Regards
>>>>>> CVP
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> *flink_job_Plan.png* (42K) Download Attachment
>>> <http://apache-flink-user-mailing-list-arc
imes.png* (65K) Download Attachment
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/attachment/9176/1/Flink-Checkpoint-Times.png>
>> *flink-qchavar-taskmanager-1-elxa1h67k32.log* (442K) Download Attachment
>> <http://apache-flink-user-mailing-list-archive.2336050.n4
Thanks, the logs were very helpful!
TL:DR - The offset committing to ZooKeeper is very slow and prevents proper
starting of checkpoints.
Here is what is happening in detail:
- Between the point when the TaskManager receives the "trigger
checkpoint" message and when the point when the KafkaSour
Hi,
I am also facing this issue, in my case the data is flowing continuously
from the Kafka source, when I increase the checkpoint interval to 6,
the data gets written to S3 sink.
Is it because some operator is taking more time for processing, like in my
case I am using a time window of 1sec.
Hi!
Let's try to figure that one out. Can you give us a bit more information?
- What source are you using for the slow input?
- How large is the state that you are checkpointing?
- Can you try to see in the log if actually the state snapshot takes that
long, or if it simply takes long for t
Hi CVP,
I'm not so much familiar with the internals of the checkpointing system,
but maybe Stephan (in CC) has an idea what's going on here.
Best, Fabian
2016-09-23 11:33 GMT+02:00 Chakravarthy varaga :
> Hi Aljoscha & Fabian,
>
> I have a stream application that has 2 stream source as belo
22 matches
Mail list logo
