Re: [DISCUSS ] add --jars to support users dependencies jars.

2022-10-26 Thread Martijn Visser
Hi Jacky Lau, Since you've sent the email to multiple mailing lists, I've decided to reply to the one that you've sent to both the Dev and User ML. > but it is not possible for platform users to create fat jars to package all their dependencies into the final jar package Can you elaborate on why

Re: configMap value error when using flink-operator?

2022-10-26 Thread Gyula Fóra
Max sense, I will fix this! Gyula On Thu, Oct 27, 2022 at 4:09 AM Biao Geng wrote: > +1 to Yang's suggestion. > > Yang Wang 于2022年10月26日周三 20:00写道: > >> Maybe we could change the values of *taskmanager.numberOfTaskSlots* and >> *parallelism.default >> *in flink-conf.yaml of Kubernetes operato

Re: Concatenating a bounded and unbounded stream

2022-10-26 Thread Jin Yi
would using a hybrid source work for you if it's the same type between the sources? https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/hybridsource/ On Wed, Oct 26, 2022 at 8:05 AM Noel OConnor wrote: > Hi, > I have a need to create a new stream from a bounded and un

Re: configMap value error when using flink-operator?

2022-10-26 Thread Biao Geng
+1 to Yang's suggestion. Yang Wang 于2022年10月26日周三 20:00写道: > Maybe we could change the values of *taskmanager.numberOfTaskSlots* and > *parallelism.default > *in flink-conf.yaml of Kubernetes operator to 1, which are aligned with > the default values in Flink codebase. > > > Best, > Yang > > Gy

Re: State Processor API - VoidNamespaceSerializer must be compatible with the old namespace serializer LongSerializer

2022-10-26 Thread Tzu-Li (Gordon) Tai
Hi Filip, I think what you are seeing is expected. The State Processor API was intended to allow access only to commonly used user-facing state structures, while Stateful Functions uses quite a bit of Flink internal features, including for its state maintenance. The list state in question in State

Concatenating a bounded and unbounded stream

2022-10-26 Thread Noel OConnor
Hi, I have a need to create a new stream from a bounded and unbounded stream i.e. create a stream populated with messages from the bounded stream and at the end of the bounded stream add messages from the unbounded stream to the new stream continuously. I know there's stream unions but the key here

Re: configMap value error when using flink-operator?

2022-10-26 Thread Yang Wang
Maybe we could change the values of *taskmanager.numberOfTaskSlots* and *parallelism.default *in flink-conf.yaml of Kubernetes operator to 1, which are aligned with the default values in Flink codebase. Best, Yang Gyula Fóra 于2022年10月26日周三 15:17写道: > Hi! > > I agree that this might be confusin

Re: State Processor API - VoidNamespaceSerializer must be compatible with the old namespace serializer LongSerializer

2022-10-26 Thread Filip Karnicki
Hi Thias Thank you for your reply. I can re-create a simplified use case at home and stick it on github if you think it will help. What I'm trying to access is pretty internal to Flink Stateful Functions. It seems that a custom operator ( https://github.com/apache/flink-statefun/blob/09a5cba521e9

Re: configMap value error when using flink-operator?

2022-10-26 Thread Gyula Fóra
Hi! I agree that this might be confusing but let me explain what happened. In the operator you can define default flink configuration. Currently it is https://github.com/apache/flink-kubernetes-operator/blob/main/helm/flink-kubernetes-operator/conf/flink-conf.yaml It contains numberOfTaskSlots=2.