[statefun] Does statefun supports query state by http endpoint?

2022-02-12 Thread casel.chen
Since statefun should bind http endpoint, I wondered if statefun supports query state by http endpoint?

[statefun] Add new state failure in Greetings example

2022-02-12 Thread casel.chen
Hello, I am trying Greeting example of Flink Stateful Functions playground. According to the README.md guide I tried to add previous login location in the state after run the example well in my laptop. I added one more state named "previous_login_location" in UserFn, and print it in GreetingsF

Re: Queryable State Deprecation

2022-02-12 Thread Frank Dekervel
Hello, This is what we did, but i'm not quite convinced that its the best way (maybe others could chime in ?). * We have a zalando postgres cluster running next to the flink cluster, so we can just use a jdbc sink for the state. In theory we should be able to switch to exactly once (we

Re: Queryable State Deprecation

2022-02-12 Thread Jatti, Karthik
Hi Frank, What sink did you end up choosing for materializing the state ? Our use case into looking at queryable state is that we have many readers and a very few writers (readers to writers ratio in the 1000s). Each consuming application (reader) needs a live view of a subset of the state and

Need help on implementing custom`snapshotState` in KafkaSource & KafkaSourceReader

2022-02-12 Thread santosh joshi
We are migrating to KafkaSource from FlinkKafkaConsumer

Re: Apache Flink - Will later events from a table with watermark be propagated and when can CURRENT_WATERMARK be null in that situation

2022-02-12 Thread M Singh
Thanks David for your insights.  Mans On Saturday, February 12, 2022, 05:26:29 AM EST, David Anderson wrote: Flink uses watermarks to indicate when a stream has become complete up through some point in time. Various operations on streams wait for watermarks in order to know when they

Re: Illegal reflective access by org.apache.flink.api.java.ClosureCleaner

2022-02-12 Thread David Anderson
You are probably running with Java 11 (with Java 8 these messages aren't produced). The Flink docs [1] say These warnings are considered harmless and will be addressed in future Flink releases. [1] https://nightlies.apache.org/flink/flink-docs-release-1.14/release-notes/flink-1.10/#java-11-su

Re: huge number of duplicate numbers after experiencing a crash loop in deserializer

2022-02-12 Thread Frank Dekervel
Hello, Small heads up, we found the cause. It had nothing to do with flink, but it was a bug in our own code. We used CEP to detect when senders would stop send messages (basically we used the timeout of a CEP pattern). And when generating these timeout messages we made inconsistent use of the

Re: Apache Flink - Will later events from a table with watermark be propagated and when can CURRENT_WATERMARK be null in that situation

2022-02-12 Thread David Anderson
Flink uses watermarks to indicate when a stream has become complete up through some point in time. Various operations on streams wait for watermarks in order to know when they can safely stop waiting for further input, and so go ahead and produce their results. These operations include event-time w