Hi Aljoscha, Which problem you are referring to ?
I am seeing unexpected stalls in between for a long time. Also one thing I have observed with FLASH_SSD_OPTIMIZED option is that it is using more amount of physical memory and not flushing the data to storage. I am trying to figure out the best possible rocksDB values for my configuration, I am currently running the job on c3.4xlarge EC2 instances Regards, Vinay Patil On Thu, Feb 16, 2017 at 4:22 PM, Aljoscha Krettek [via Apache Flink User Mailing List archive.] <ml-node+s2336050n11668...@n4.nabble.com> wrote: > [hidden email] <http:///user/SendEmail.jtp?type=node&node=11668&i=0> and > [hidden > email] <http:///user/SendEmail.jtp?type=node&node=11668&i=1> could this > be the same problem that you recently saw when working with other people? > > On Wed, 15 Feb 2017 at 17:23 Vinay Patil <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=11668&i=2>> wrote: > >> Hi Guys, >> >> Can anyone please help me with this issue >> >> Regards, >> Vinay Patil >> >> On Wed, Feb 15, 2017 at 6:17 PM, Vinay Patil <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=11668&i=3>> wrote: >> >> Hi Ted, >> >> I have 3 boxes in my pipeline , 1st and 2nd box containing source and s3 >> sink and the 3rd box is window operator followed by chained operators and a >> s3 sink >> >> So in the details link section I can see that that S3 sink is taking time >> for the acknowledgement and it is not even going to the window operator >> chain. >> >> But as shown in the snapshot ,checkpoint id 19 did not get any >> acknowledgement. Not sure what is causing the issue >> >> >> Regards, >> Vinay Patil >> >> On Wed, Feb 15, 2017 at 5:51 PM, Ted Yu [via Apache Flink User Mailing >> List archive.] <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=11668&i=4>> wrote: >> >> What did the More Details link say ? >> >> Thanks >> >> > On Feb 15, 2017, at 3:11 AM, vinay patil <[hidden email] >> <http:///user/SendEmail.jtp?type=node&node=11641&i=0>> wrote: >> > >> > Hi, >> > >> > I have kept the checkpointing interval to 6secs and minimum pause >> between >> > checkpoints to 5secs, while testing the pipeline I have observed that >> that >> > for some checkpoints it is taking long time , as you can see in the >> attached >> > snapshot checkpoint id 19 took the maximum time before it gets failed, >> > although it has not received any acknowledgements, now during this >> 10minutes >> > the entire pipeline did not make any progress and no data was getting >> > processed. (For Ex : In 13minutes 20M records were processed and when >> the >> > checkpoint took time there was no progress for the next 10minutes) >> > >> > I have even tried to set max checkpoint timeout to 3min, but in that >> case as >> > well multiple checkpoints were getting failed. >> > >> > I have set RocksDB FLASH_SSD_OPTION >> > What could be the issue ? >> > >> > P.S. I am writing to 3 S3 sinks >> > >> > checkpointing_issue.PNG >> > <http://apache-flink-user-mailing-list-archive.2336050. >> n4.nabble.com/file/n11640/checkpointing_issue.PNG> >> > >> > >> > >> > -- >> > View this message in context: http://apache-flink-user- >> mailing-list-archive.2336050.n4.nabble.com/Checkpointing- >> with-RocksDB-as-statebackend-tp11640.html >> > Sent from the Apache Flink User Mailing List archive. mailing list >> archive at Nabble.com. >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the discussion >> below: >> http://apache-flink-user-mailing-list-archive.2336050. >> n4.nabble.com/Checkpointing-with-RocksDB-as-statebackend- >> tp11640p11641.html >> >> To start a new topic under Apache Flink User Mailing List archive., email >> [hidden >> email] <http:///user/SendEmail.jtp?type=node&node=11668&i=5> >> >> To unsubscribe from Apache Flink User Mailing List archive., click here. >> NAML >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> >> > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-mailing-list-archive.2336050. > n4.nabble.com/Checkpointing-with-RocksDB-as-statebackend- > tp11640p11668.html > To start a new topic under Apache Flink User Mailing List archive., email > ml-node+s2336050n1...@n4.nabble.com > To unsubscribe from Apache Flink User Mailing List archive., click here > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx> > . > NAML > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Checkpointing-with-RocksDB-as-statebackend-tp11640p11672.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
