What kind of problem are we talking about? S3 related or RocksDB related. I am not aware of problems with RocksDB per se. I think seeing logs for this would be very helpful.
> Am 16.02.2017 um 11:56 schrieb Aljoscha Krettek <aljos...@apache.org>: > > +Stefan Richter <mailto:s.rich...@data-artisans.com> and +Stephan Ewen > <mailto:se...@apache.org> could this be the same problem that you recently > saw when working with other people? > > On Wed, 15 Feb 2017 at 17:23 Vinay Patil <vinay18.pa...@gmail.com > <mailto:vinay18.pa...@gmail.com>> wrote: > Hi Guys, > > Can anyone please help me with this issue > > Regards, > Vinay Patil > > On Wed, Feb 15, 2017 at 6:17 PM, Vinay Patil <vinay18.pa...@gmail.com > <mailto:vinay18.pa...@gmail.com>> wrote: > Hi Ted, > > I have 3 boxes in my pipeline , 1st and 2nd box containing source and s3 sink > and the 3rd box is window operator followed by chained operators and a s3 sink > > So in the details link section I can see that that S3 sink is taking time for > the acknowledgement and it is not even going to the window operator chain. > > But as shown in the snapshot ,checkpoint id 19 did not get any > acknowledgement. Not sure what is causing the issue > > Regards, > Vinay Patil > > On Wed, Feb 15, 2017 at 5:51 PM, Ted Yu [via Apache Flink User Mailing List > archive.] <ml-node+s2336050n11641...@n4.nabble.com > <mailto:ml-node+s2336050n11641...@n4.nabble.com>> wrote: > What did the More Details link say ? > > Thanks > > > On Feb 15, 2017, at 3:11 AM, vinay patil <[hidden email] > > <http://user/SendEmail.jtp?type=node&node=11641&i=0>> wrote: > > > > Hi, > > > > I have kept the checkpointing interval to 6secs and minimum pause between > > checkpoints to 5secs, while testing the pipeline I have observed that that > > for some checkpoints it is taking long time , as you can see in the > > attached > > snapshot checkpoint id 19 took the maximum time before it gets failed, > > although it has not received any acknowledgements, now during this > > 10minutes > > the entire pipeline did not make any progress and no data was getting > > processed. (For Ex : In 13minutes 20M records were processed and when the > > checkpoint took time there was no progress for the next 10minutes) > > > > I have even tried to set max checkpoint timeout to 3min, but in that case > > as > > well multiple checkpoints were getting failed. > > > > I have set RocksDB FLASH_SSD_OPTION > > What could be the issue ? > > > > P.S. I am writing to 3 S3 sinks > > > > checkpointing_issue.PNG > > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/n11640/checkpointing_issue.PNG > > > > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/n11640/checkpointing_issue.PNG>> > > > > > > > > > > -- > > View this message in context: > > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Checkpointing-with-RocksDB-as-statebackend-tp11640.html > > > > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Checkpointing-with-RocksDB-as-statebackend-tp11640.html> > > Sent from the Apache Flink User Mailing List archive. mailing list archive > > at Nabble.com. > > > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Checkpointing-with-RocksDB-as-statebackend-tp11640p11641.html > > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Checkpointing-with-RocksDB-as-statebackend-tp11640p11641.html> > To start a new topic under Apache Flink User Mailing List archive., email > ml-node+s2336050n1...@n4.nabble.com > <mailto:ml-node%2bs2336050n1...@n4.nabble.com> > To unsubscribe from Apache Flink User Mailing List archive., click here > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>. > NAML > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
