Re: Re: Re: Flink kafka connector for v 1.19.0

2024-05-17 Thread Niklas Wilcke
ues.apache.org/jira/browse/FLINK-35138 > [2] https://lists.apache.org/thread/7shs2wzb0jkfdyst3mh6d9pn3z1bo93c > > Niklas Wilcke mailto:niklas.wil...@uniberg.com>> > 于2024年5月16日周四 22:04写道: >> Hi Ahmed, >> >> are you aware of a blocker? I'm also a bit co

Re: [EXTERNAL]Re: Flink kafka connector for v 1.19.0

2024-05-16 Thread Niklas Wilcke
Hi Ahmed, are you aware of a blocker? I'm also a bit confused that after Flink 1.19 being available for a month now the connectors still aren't. It would be great to get some insights or maybe a reference to an issue. From looking at the Github repos and the Jira I wasn't able to spot something

Re: [EXTERNAL]Re: Pyflink Performance and Benchmark

2024-04-15 Thread Niklas Wilcke
> Best, > Zhanghao Chen > > From: Niklas Wilcke > Sent: Monday, April 15, 2024 15:17 > To: user > Subject: Pyflink Performance and Benchmark > > Hi Flink Community, > > I wanted to reach out to you to get some input about Pyflink performance. Are > there any resou

Pyflink Performance and Benchmark

2024-04-15 Thread Niklas Wilcke
Hi Flink Community, I wanted to reach out to you to get some input about Pyflink performance. Are there any resources available about Pyflink benchmarks and maybe a comparison with the Java API? I wasn't able to find something valuable, but maybe I missed something? I am aware that benchmarking

Re: [EXTERNAL]Re: Flink Kubernetes Operator - Deadlock when Cluster Cleanup Fails

2024-02-13 Thread Niklas Wilcke
addition to improving logging we could also increase the default timeout and >> if we could actually do something on the timeout that would be even better. >> >> Please open the JIRA ticket and if you have time to work on these >> improvements I will assign it to y

Re: [EXTERNAL]Re: Flink Kubernetes Operator - Deadlock when Cluster Cleanup Fails

2024-02-13 Thread Niklas Wilcke
hing on the timeout that would be even better. > > Please open the JIRA ticket and if you have time to work on these > improvements I will assign it to you. > > Cheers > Gyula > > On Mon, Feb 12, 2024 at 11:59 PM Niklas Wilcke <mailto:niklas.wil...@uniberg.com>&g

Flink Kubernetes Operator - Deadlock when Cluster Cleanup Fails

2024-02-12 Thread Niklas Wilcke
Hi Flink Kubernetes Operator Community, I hope this is the right way to report an issue with the Apache Flink Kubernetes Operator. We are experiencing problems with some streaming job clusters which end up in a terminated state, because of the operator not behaving as expected. The problem is t

Re: [EXTERNAL]Re: Next Release of the Flink Kubernetes Operator

2023-10-17 Thread Niklas Wilcke
/flink-kubernetes-operator/127962962?tag=3f0dc2e > > On Mon, Oct 16, 2023 at 5:41 PM Niklas Wilcke <mailto:niklas.wil...@uniberg.com>> wrote: >> Hi Flink Community, >> >> we are waiting for the next release of the Flink Kubernetes Operator, >> because we

Next Release of the Flink Kubernetes Operator

2023-10-16 Thread Niklas Wilcke
Hi Flink Community, we are waiting for the next release of the Flink Kubernetes Operator, because we are experiencing problems with loosing the HA metadata similar to FLINK-33011 [0]. Since the problem is already fixed and also backported to the 1.6 branch [1], my question would be whether we c

Re: KafkaSource and Event Time in Message Payload

2022-12-15 Thread Niklas Wilcke
nce/display/FLINK/FLIP-217%3A+Support+watermark+alignment+of+source+splits > > On Thu, Dec 8, 2022 at 6:21 PM Niklas Wilcke <mailto:niklas.wil...@uniberg.com>> wrote: >> Hi Flink Community, >> >> I have a few questions regarding the new KafkaSource and event time,

KafkaSource and Event Time in Message Payload

2022-12-08 Thread Niklas Wilcke
Hi Flink Community, I have a few questions regarding the new KafkaSource and event time, which I wasn't able to answer myself via checking the docs, but please point me to the right pages in case I missed something. I'm not entirely whether my knowledge entirely holds for the new KafkaSource, b

Verifying max-parallelism value

2021-09-02 Thread Niklas Wilcke
Hi Flink community, most likely I'm missing something but I failed to verify the setting of the max-parallelism (# key groups). Is there a way to check the value for a job? I checked the following places without finding it. 1. Flink UI: Job Configuration 2. Flink UI: SubTasks of a Job 3. Taskma

Re: Production Grade GitOps Based Kubernetes Setup

2021-08-09 Thread Niklas Wilcke
efore doing the rollover). > 3. At it's current state it looks more like a side project than an actively > maintained operator. > 4. Ververica is definitely an option, we haven't tested their operator, not > sure about the maturity level yet. > > I think a Flink communit

Re: Production Grade GitOps Based Kubernetes Setup

2021-08-09 Thread Niklas Wilcke
le once) > - delay with Flink Versions > > One year later I still think there is no perfect solution for managing > Flink on K8s, but for us Ververica was the closest match. > > Regards, > Maciek > > pt., 6 sie 2021 o 13:49 Niklas Wilcke napisał(a): >> >>

Production Grade GitOps Based Kubernetes Setup

2021-08-06 Thread Niklas Wilcke
olution like "Ververica Platform Community Edition" a solution for a production environment or will it definitely lack features I need? Any information or feedback is highly appreciated. Thank you very much in advance. Kind Regards, Niklas Wilcke [1] https://ci.apache.org/pr

Re: PyFlink Table API and UDF Limitations

2020-11-27 Thread Niklas Wilcke
ease-1.11/dev/table/tableApi.html#row-based-operations > > <https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/tableApi.html#row-based-operations> > > Best, > Xingbo > > Niklas Wilcke mailto:niklas.wil...@uniberg.com>> > 于2020年11月26日周四 下午5:1

Re: PyFlink Table API and UDF Limitations

2020-11-26 Thread Niklas Wilcke
gt; really great if I would be able to return a list with multiple fields. >> Currently I'm just putting multiple value into a single VARCHAR, but that >> means the information needs to be extracted later on. Maybe you have an idea >> how to avoid that. > > Currently

Re: State of Machine Learning with Flink and especially FLIP-39

2020-11-20 Thread Niklas Wilcke
e been done so far in the DataSet API, > which is going to deprecated in the next few releases in favor of the unified > DataStream API with bounded streams. I expect efforts for native > implementations to be picked up once DataSet is fully replaced to avoid > doubling the work. On

State of Machine Learning with Flink and especially FLIP-39

2020-11-19 Thread Niklas Wilcke
Hi Flink-Community, I'm digging through the history of FlinkML and FLIP-39 [0]. What I understood so far is that FlinkML has been removed in 1.9, because it got unmaintained. I'm not really able to find out whether FLIP-39 and providing a replacement for FlinkML is currently worked on. The Umbre

Re: PyFlink Table API and UDF Limitations

2020-11-13 Thread Niklas Wilcke
h as map, flat_map, etc. > > [1] > https://ci.apache.org/projects/flink/flink-docs-master/dev/python/datastream_tutorial.html > > <https://ci.apache.org/projects/flink/flink-docs-master/dev/python/datastream_tutorial.html> >> 在 2020年11月12日,下午7:46,Niklas Wilcke >

Re: PyFlink Table API and UDF Limitations

2020-11-12 Thread Niklas Wilcke
kDev/building.html#build-pyflink > > <https://ci.apache.org/projects/flink/flink-docs-master/flinkDev/building.html#build-pyflink> > > Regards, > Dian > >> 在 2020年11月11日,下午9:03,Niklas Wilcke > <mailto:niklas.wil...@uniberg.com>> 写道: >> >> Hi Flin

PyFlink Table API and UDF Limitations

2020-11-11 Thread Niklas Wilcke
Hi Flink Community, I'm currently trying to implement a parallel machine learning job with Flink. The goal is to train models in parallel for independent time series in the same data stream. For that purpose I'm using a Python library, which lead me to PyFlink. Let me explain the use case a bit