Re: 404 Jar File Not Found w/ Web Submit Disabled

Hi, Currently you can upload a jar job to a flink session cluster, or submit a job graph to the session cluster with rest api, for example, submit sql jobs with jdbc driver to sql-gateway, then the gateway will build job graph and submit it to the session cluster via rest endpoint. If you configu

Re: Flink SQL query with window-TVF fails

Hi, Pouria Flink SQL uses the calcite to parse SQL, this is the calcite limitation, the minimum precision it supports is Second [1]. [1] https://github.com/apache/calcite/blob/main/core/src/main/codegen/templates/Parser.jj#L5067 Best, Ron Pouria Pirzadeh 于2023年8月15日周二 08:09写道： > I am trying t

Re: Recommendations on using multithreading in flink map functions in java

Hi, Vignesh Flink is a distributed parallel computing framework, each MapFunction is actually a separate thread. If you want more threads to process the data, you can increase the parallelism of the MapFunction without having to use multiple threads in a single MapFunction, which in itself violate

Flink SQL query with window-TVF fails

I am trying to run a window aggregation SQL query (on Flink 1.16) with Windowing TVF for a TUMBLE window with a size of 5 Milliseconds it seems Flink does not let a window size use a time unit smaller than seconds. Is that correct? (The documentation

Recommendations on using multithreading in flink map functions in java

Hello All, *Problem statement * For a given element, I have to perform multiple(lets say N) operations on it. All the N operations are independent of each other. And for achieving lowest latency, I want to do them concurrently. I want to understand what's the best way to perform it in flink?. I u

Re: [DISCUSS] Status of Statefun Project

I created a pull request for this: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1 by galenwarren · Pull Request #331 · apache/flink-statefun (github.com) . JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1 - ASF JIRA

Re: 404 Jar File Not Found w/ Web Submit Disabled

Hi, Just to add, when I set back to "true" the web.ui submit property, that is when the rest endpoint /jars/upload worked again. But in the documentation reference: https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/ Disabling the UI doesnt disable the endpoint. Is this

Re: Question about serialization of java.util classes

Hello, AFAIK you cannot avoid TypeInformationFactory due to type erasure, nothing Flink can do about that. Here's an example of helper classes I've been using to support set serde in Flink POJOs, but note that it's hardcoded for LinkedHashSet, so you would have to create different implementations

Re: Streaming join performance

Привет Артем! Are your tables backed by Kafka? If - yes, what if you use upsert-kafka connector from Table API , does it help to reduce the number of records in each subsequent join operator? I wrote a blog-p

Re: Question about serialization of java.util classes

Hi, Here's a minimal example using an ArrayList, a HashSet, and a TreeSet: ``` package com.example; import java.util.ArrayList; import java.util.HashSet; import java.util.TreeSet; import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; public class App { public static cla

Re: Question about serialization of java.util classes

Hi Saleh, If you could show us the minimal code example of the issue (event classes), I think someone could help you to solve it. Best regards, Alexey On Mon, Aug 14, 2023 at 9:23 AM wrote: > Hi, > > According to this blog post > https://flink.apache.org/2020/04/15/flink-serialization-tuning-v

404 Jar File Not Found w/ Web Submit Disabled

Hi, I disabled the web.ui.submit=false, after that uploading jar files via rest endpoint is now throwing 404. In the documentation it says: "Even it is disabled sessions clusters still accept jobs through REST requests (Http calls). This flag only guards the feature to upload jobs in the UI" I a

Re: Dependency injection framework for flink

I would agree with Ron. If you have a chance to use Scala, then it is much easier to compose Flink process functions (or what have you) into a data stream. Simple Functional Programming power. Coming from a Java background into the Scala ecosystem sometime ago, I was just surprised that proper lan

Re: Conversion expects insert-only records but DataStream API record contains: UPDATE_BEFORE

Hi, Changelog mode is the concept of the table API. You can get a changelog stream from StreamTableEnvironment#fromChangelogStream. For kafka connector, its changelog mode depends on the used format. Best, Hang liu ron 于2023年8月13日周日 22:06写道： > Hi, > > After deep dive into the source code, I gu

Re: Question about serialization of java.util classes

Hi, According to this blog post https://flink.apache.org/2020/04/15/flink-serialization-tuning-vol.-1-choosing-your-serializer-if-you-can/#pojoserializer The "Must be proc