Re: Queryable State Deprecation

2022-03-11 Thread Ron Crocker
Hi Dawid - I’m pretty keen on keeping it alive. Do we have a sense of what it would take to get it “to a production ready state?” Thanks! Ron > On Feb 4, 2022, at 5:06 AM, Dawid Wysakowicz wrote: > > Hi Karthik, > > The reason we deprecated it is because we lacked committers who could spend

Re: Checkpointing in StateFun

2022-03-11 Thread Seth Wiesman
I assume you are talking about the checkpointing in the feedback package? StateFun only relies on Flink checkpointing for fault tolerance. All state is stored in standard checkpoint / savepoints and can be used to restore from failure, upgrade a job, rescale, etc. Just like any other snapshot. St

Slowness using GenericWriteAheadSink

2022-03-11 Thread Sandys-Lumsdaine, James
Hello, We are using the GenericWriteAheadSink to buffer up values to then send to a SQL Server database with a fast bulk copy upload. However, when I watch my process running it seems to be a huge amount of time iterating the Iterable provided to the sendValues() method. It takes such a long ti

Re: Add libatomic1 to flink image

2022-03-11 Thread Konstantin Knauf
Moving dev@ to bcc, adding user@ Hi Julius, the recommended approach would be to build your own Docker images from the official images along the lines of FROM apache/flink:1.14.3 RUN apt-get install -y libatomic1 Cheers, Konstantin On Fri, Mar 11, 2022 at 11:07 AM Almeida, Julius wrote: >

Checkpointing in StateFun

2022-03-11 Thread Christopher Gustafson
Hi, I am doing some experiments with StateFun and different Flink state backends. I was looking through the StateFun source code and saw that it is doing some own logic when it comes to checkpointing. My initial understanding was that checkpointing was implicitly leveraged with Flink. I did no

Re: Flink Statefun and Feature computation

2022-03-11 Thread Igal Shilman
Hello Federico, Yes you are correct, the type of the storage engine is indeed configured via the flink-yaml.conf, the state engines you have mentioned are the ones that are actively mantinted and configured by the community for a long time. I'm not sure however what are the requirements for imple