log :: Checkpoint 58 of job 0efaa0e6db5c38bec81dfefb159402c0 expired before completing. I have a use case where i need to do the checkpointing frequently .
i am using Kafka to read stream and making a window of 1 hour , which is having 50gb data always and it can be more than that . i have seen there is no back pressure . Thanks Yubraj Singh On Wed, Sep 19, 2018 at 12:07 PM Jörn Franke <jornfra...@gmail.com> wrote: > What do the logfiles say? > > How does the source code looks like? > > Is it really needed to do checkpointing every 30 seconds? > > On 19. Sep 2018, at 08:25, yuvraj singh <19yuvrajsing...@gmail.com> wrote: > > Hi , > > I am doing checkpointing using s3 and rocksdb , > i am doing checkpointing per 30 seconds and time out is 10 seconds . > > most of the time its failing by saying Failure Time: 11:53:17Cause: > Checkpoint expired before completing . > I increases the timeout as well still it not working for me . > > please suggest . > > Thanks > Yubraj Singh > >
