RE: Identifying a flink dashboard

Hi Mike, Let me sketch it: * The trick I use (no idea if it is wise or not 😊 ) is to have nginx-ingress set up and then specify a service selecting the nginx…controller pods [1] * You don’t need to bind to the node address (see externalIPs), you could much the same port-forward this se

Re: Identifying a flink dashboard

G'day, The flink and the dashboard are running in k8s and I am not on the same network. We don't have a VPN into the cluster. (Don't ask) I am not sure how I would access the dashboard without having a port forward. On 28/06/2023 14:39, Schwalbe Matthias wrote: Good Morning Mike, As a qu

Re: [Slack] Request to upload new invitation link

Hi, Stephen. Welcome to join Flink Slack channel. Here's my invitation link: https://join.slack.com/t/apache-flink/shared_invite/zt-1y7kmx7te-zUg1yfLdGu3Th9En_p4n~g Best regards, Yuxia 发件人: "Stephen Chu" 收件人: "User" 抄送: "Satyam Shanker" , "Vaibhav Gosain" , "Steve Jiang" 发送时间: 星期四,

Re: 回复: Questions regarding adaptive scheduler with YARN and application mode

Thank you both. Looks like with the upcoming Flink 1.18 release, we should build an auto-scaler service to monitor the job and properly adjust the allocated resource from YARN. Leon On Wed, Jun 28, 2023 at 9:08 AM Madan D wrote: > Hello Leon, > > As described by Chen below Adaptive Scheduler d

Re: Very long launch of the Flink application in BATCH mode

Hi Shammon, When I set log.level=DEBUG I have no more logs except *2023-06-21 14:51:30,921 DEBUG org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] - Trigger heartbeat request.* Job freezes on stream graph generation. In STREAMING mode the job starts fast without same probl

[Slack] Request to upload new invitation link

Hi there, I'd love to join the Flink Slack channel, but it seems the link is outdated: https://join.slack.com/t/apache-flink/shared_invite/zt-1thin01ch-tYuj6Zwu8qf0QsivHY0anw Would someone be able to update or send me a new invite link? Thanks, Stephen

Re: 回复: Questions regarding adaptive scheduler with YARN and application mode

Hello Leon, As described by Chen below Adaptive Scheduler doesn't perform auto scale a Flink Job other than allocating the requested slots based on availability. Recently we implemented this with EMR managed scaling by combining adaptive scheduler since there's no direct support of auto scaling

回复: Questions regarding adaptive scheduler with YARN and application mode

Hi Leon, Adaptive scheduler alone cannot autoscale a Flink job. It simply adjusts the parallelism of a job based on available slots [1]. To autoscale a job, we further need a policy to suggest the recommended resources for the job and a mechanism to adjust the allocated resources of the job (ak

Re: Kafka source with idleness and alignment stops consuming

Hello, just for completeness, I don't see the problem if I assign a different alignment group to each source, i.e. using only split-level watermark alignment. Regards, Alexis. Am Mi., 28. Juni 2023 um 08:13 Uhr schrieb haishui : > Hi, > I have the same trouble. This is really a bug. > `shouldWa

Re: how to get blackhole connector jar

Hi, longfeng, I check the blackhole connector document[1] and the blackhole connector is a build-in connector. If you can not find this connector in your flink, maybe you should check the `flink-table-api-java-bridge` jar to find the `BlackHoleTableSinkFactory`[2]. Best, Hang [1] https://nightli

how to get blackhole connector jar

hi guys， when using alibaba flink (version flink 1.12) to running nexmark's query0 , it failed blackhole is not a supported sink connector. so how can i upload connector-blackhole like nexmark connector? where to download it? thanks