Re: onTimer() of a KeyedProcessFunction stops getting triggered after a while

After fixing your negative timestamp bug, can the timer be triggered? > On 23 Mar 2022, at 2:39 AM, Binil Benjamin wrote: > > Here are some more findings as I was debugging this. I peeked into the > snapshot to see the current values in "_timer_state/processing_user-timers" > and here is ho

FileSystem format

Hi team, So basically, when I use Flink Table API to generate the files and store in S3. The format files will be like this part-0d373ee1-d594-40b1-a3cf-8fa895260980-0-0. So my question is is there any way that we can config this files names (by adding the last_modified_value) to this files name ?B

Re: Flink plugin can't get access to some of the classes in /opt/flink/lib folder

Hi! You can set the classloader.parent-first-patterns.additional configuration to load additional classes from the lib directory. See [1] for more details. What classes are causing the ClassNotFoundException? From the default value of classloader.parent-first-patterns.default all classes in org.a

Flink plugin can't get access to some of the classes in /opt/flink/lib folder

Hi Flink Community, Recently I implemented a new metrics reporter to report Flink runtime metrics to a Kafka topic. After building the project, I move my reporter jar file from my build folder to the plugin folder: COPY build/flink-diagnostics-message-reporter-${VERSION}.jar ${FLINK_HOME}/plug

Clust rconfiguration for network-intensive Flink job

Hi, folks! I am running Flink Streaming job in mode=Batch on EMR. The job has following stages: 1. Read from MySQL 2. KeyBy user_id 3. Reduce by user_id 4. Async I/O enriching from Redis 5. Async I/O enriching from other Redis 6. Async I/O enriching from REST #1 7.

Using Amazon EC2 Spot instances with Flink

Hello, folks! I was wondering if there are some good articles on how to use EC2 Spot instances with Flink? I would appreciate your help! Thank you! Kind Regards, Valeriia

Flink OOM issue

Hello, We encountered OOM in the pipeline sink step on a consistent basis. How can we debug this? Below is the error: java.lang.OutOfMemoryError: Metaspace. The metaspace out-of-memory error has occurred. This can mean two things: either the job requires a larger size of JVM metaspace to load cl

Re: onTimer() of a KeyedProcessFunction stops getting triggered after a while

Here are some more findings as I was debugging this. I peeked into the snapshot to see the current values in "_timer_state/processing_user-timers" and here is how they look: Timer{timestamp=-9223372036854715808, key=(FFX22...), namespace=VoidNamespace} Timer{timestamp=-9223372036854715808, key=(FF

Re: scala shell not part of 1.14.4 download

Many thanks. In the linked discussion it sounds like a move - not a delete. However, no destination is named. Is there currently any moved version of the scala shell available elsewhere? Best, Georg Am Mo., 21. März 2022 um 09:26 Uhr schrieb Martijn Visser < martijnvis...@apache.org>: > Hi Geor

Connection warnings when running Flink statefun

Hi team, I'm trying to run a flink-statefun application (version 3.2.0) on my local machine. The application is a pipeline consisting of multiple services that communicate to each other via sending http requests served by aiohttp. I am using a single job manager and a single task manager. When I r

Python UDF Gauge Metrics not working & error log on Vectorized UDF

Hello, I try to run my flink java job using Python UDF, but i have some doubt using gauge metrics and vectorized udf on it. 1. i try to create a simple udf like this example on the docs https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/table/metrics/#gauge. i al

Re: [External] Re: Potential Bug with Date Serialization for Table Stream

Hi Martijn, Do you know what could be causing this issue given our Flink version? Is this possibly a bug with that version? Thanks, Tom On Thu, Mar 17, 2022 at 9:59 AM Tom Thornton wrote: > Hi Martijn, > > We are using 1.11.6. > > Thank you for the help. > > On Thu, Mar 17, 2022 at 1:37 AM Mar

[ANNOUNCE] Call for Presentations is open for Flink Forward San Francisco 2022 in-person!

Hi everyone, We’re very excited to announce our Call for Presentations for Flink Forward San Francisco 2022! If you have an inspiring Apache Flink use case, real-world application, or best practice, Flink Forward is the platform for you to share your experiences. https://www.flink-forward.or

Asking about the partition files

Hi team, So basically, when I use Flink Table API to generate the files and store in S3. The format files will be like this part-0d373ee1-d594-40b1-a3cf-8fa895260980-0-0. So my question is is there any way that we can config this files names (by adding the last_modified_value) to this files name ?B

Re: how to set kafka sink ssl properties

Hello Matthias and others I am trying to configure a Kafka Sink with SSL properties as shown further below. But in the logs I see warnings: 2022-03-21 12:30:17,108 WARN org.apache.kafka.clients.admin.AdminClientConfig [] - The configuration 'group.id' was supplied but isn't a known