Re: Debug CEP Patterns

2023-04-18 Thread Biao Geng
Hi Ana, Thanks for the codes. I just want to share my own experience when debugging CEP patterns: 1. It should work when adding breakpoints in IntelliJ. But you should add the breakpoint on the filter function like the call of isBetweenHoursRange or the line65 in your gist. The reason is that the p

Re: Flink not releasing the reference to a deleted jar file

2023-04-18 Thread neha goyal
Adding to the above query, I have tried dropping the tables and the function as well but no luck. On Wed, Apr 19, 2023 at 11:01 AM neha goyal wrote: > Hello, > > I am attaching a sample code and screenshot where Flink is holding the > reference to a jar file even after I close the streamExecutio

Re: Opensearch connector and oauth

2023-04-18 Thread Michael Hempel Jørgensen
Hi Andriy, we are currently use the OpensearchSink[1] connector, as we assumed that that was the one to use going forwards. We are not quite sure what the difference is and if any will become deprecated soon? Regards, Michael [1] https://github.com/apache/flink-connector-opensearch/blob/m

Flink not releasing the reference to a deleted jar file

2023-04-18 Thread neha goyal
Hello, I am attaching a sample code and screenshot where Flink is holding the reference to a jar file even after I close the streamExecutionEnvironment. Due to this, the deleted file is not getting cleaned up from the disk and we are getting disc space alerts. When we restart our application then

Re: Debug CEP Patterns

2023-04-18 Thread Ana Gómez González
Here's a link for the previous code in a gist so you don't struggle with the format. Sorry. https://gist.github.com/angoglez/d9eb6e12f259aba306387b5c23488fb4 >

Debug CEP Patterns

2023-04-18 Thread Ana Gómez González
Hello! What's the best way of debugging a CEP pattern/stream? Basically, I have a Flink (Scala) program that consumes events from RabbitMQ. The source is working ok because I can see the inputStream printed. Then, I defined one pattern, created a patternStream, and finally an output stream with a

Re: Task Failure Strategy for Adaptive Scheduler

2023-04-18 Thread Talat Uyarer via user
Hi David, Yes We have multiple disjoint DAGs in one job. We want better CPU utilization. Open Source Flink has a scheduling issue with those types of jobs. I made a fix on 1.13 with AS. Now we are scheduling evenly for all DAGs. However somehow when we get an exception on a DAG we dont want to a

Re: Opensearch connector and oauth

2023-04-18 Thread Andriy Redko
Hi Michael, Indeed the Opensearch REST client supports only basic auth at the moment. I don't know which sink are you using, but the streaming/connectors/opensearch/OpensearchSink provides the capability to configure the underlying client using RestClientFactory [1] so you could add the reques

Re: Opensearch connector and oauth

2023-04-18 Thread Martijn Visser
Hi Michael, I'm looping in Andrey since he has worked a lot on the Opensearch connector. A contribution is very welcome in case this can be improved. Best regards, Martijn On Tue, Apr 18, 2023 at 8:45 AM Michael Hempel Jørgensen wrote: > Hi, > > we need to use OAuth2 (Client Credentials Flo

Re: Task Failure Strategy for Adaptive Scheduler

2023-04-18 Thread David Morávek
> Our DAG has multiple sources which are not connected to each other. To double-check, are you saying the job consists of multiple disjoint DAGs? > Do you think somehow the adaptive scheduler supports region pipelines for streaming jobs ? It's doable but might not be straightforward since the A

Re: 取消订阅

2023-04-18 Thread Leonard Xu
可以发送任意内容的邮件到 user-unsubscr...@flink.apache.org 取消订阅来自 user@flink.apache.org 邮件列表的邮件,其他邮件列表的订阅和退订管理也类似,邮件列表的订阅管理,可以参考[1] 祝好, Leonard Xu [1] https://flink.apache.org/community.html#how-to-subscribe-to-a-mailing-list > On Apr 18, 2023, at 2:53 PM, wangw...@sina.cn wrote: > > 取消订阅