Instructions / steps regarding contributing : flink-training

Hi I want to add / modify a few unit tests for flink-training. I have came across following instructions : Contributing I am not sure about the formal process of submitting the change or any JIRA ticket etc requirements.

Re: Incompatible KafkaProducer version

Thanks for your suggestion Martjin. I am in the process of upgrading but this is kind of a blocker for coming to a sane state before proceeding. Is there any generic guide for version upgrading ? On Mon, Feb 6, 2023 at 11:38 AM Martijn Visser wrote: > Hi Milind Vaidya, > > I wou

Incompatible KafkaProducer version

Hi Getting the following exception while trying to upgrade Kafka Libraries Flink version : 1.9.3 Old Kafka version : 2.2.1.8 New Kafka version : 2.4.1.4 Found a ticket : https://issues.apache.org/jira/browse/FLINK-9690 But that seems to be between 0.11 to 2.0.0. Cause : Caused by: java.lang.No

Re: Failing to build Flink 1.9 using Scala 2.12

for more info. > > Regards, > David > > On Fri, Dec 23, 2022 at 6:55 AM Milind Vaidya wrote: > >> Hi >> >> First of all, I do understand that I am using a very old version. But as >> of now the team can not help it. We need to move to Scala 2.12 first and >

Failing to build Flink 1.9 using Scala 2.12

Hi First of all, I do understand that I am using a very old version. But as of now the team can not help it. We need to move to Scala 2.12 first and then we will move forward towards the latest version of Flink. I have added following things to main pom.xml 2.11.12 2.11 Under Scala-2.11 remov

Re: Failing to compile Flink 1.9 with Scala 2.12

nded to use the > latest stable version 1.15.1. > > > Best, > Weihua > > > On Thu, Aug 18, 2022 at 5:36 AM Milind Vaidya wrote: > >> Hi >> >> Trying to compile and build Flink jars based on Scala 2.12. >> >> Settings : >> Java 8 >> M

Failing to compile Flink 1.9 with Scala 2.12

Hi Trying to compile and build Flink jars based on Scala 2.12. Settings : Java 8 Maven 3.6.3 / 3.8.6 Many online posts suggest using Java 8 which is already in place. Building using Jenkins. Any clues as to how to get rid of it? net.alchim31.maven scala-maven-plugin 3.3.2 -nobootcp -Xs

Re: Flink 1.12 StreamRecordQueueEntry is not public class

me retrofit work on flink, and it's a fast way to tag it as public and > rebuild flink just for customization. > > At 2022-06-24 06:27:43, "Milind Vaidya" wrote: > > Hi > > I am trying to upg

Flink 1.12 StreamRecordQueueEntry is not public class

Hi I am trying to upgrade the version to 1.12. Some of the old code is using org.apache.flink.streaming.api.operators.async.queue.StreamRecordQueueEntry; This is no longer public in 1.12. Any pointers as to how to work around this? Thanks, Milind

Re: Issues with forwarding environment variables

String serviceName = System.getenv("SERVICE_NAME"); > Is it a scala object? If so, it can be initialized before any > properties are set. > What happens if the variable/property is read later at run time? > > Regards, > Roman > > On Thu, May 20, 2021 at 10:41 PM Milind Vai

Re: Issues with forwarding environment variables

.opts.taskmanager: -DSERVICE_NAME=... > In the application: System.getProperty("SERVICE_NAME"); > > Regards, > Roman > > On Thu, May 20, 2021 at 9:50 PM Milind Vaidya wrote: > > > > > > Hi Roman, > > > > I have

Re: Issues with forwarding environment variables

you please share the relevant parts of your flink-conf.yaml? > > Regards, > Roman > > On Thu, May 20, 2021 at 9:13 PM Milind Vaidya wrote: > > > > Hi > > > > Need to forward a few env variables to Job and Task manager. > > I am running jobs in Yarn c

Issues with forwarding environment variables

Hi Need to forward a few env variables to Job and Task manager. I am running jobs in Yarn cluster I was referring to this : Forwarding I also found Stack Overflow

Re: MemoryStateBackend Issue

u could achieve the former one by killing the TaskManager instance or by > throwing a custom failure as part of your job pipeline. The latter one can > be achieved by using stop-with-savepoint instead of canceling the job. > > Matthias > > On Fri, Apr 23, 2021 at 9:31 PM Milind Vaidya

Re: MemoryStateBackend Issue

e might have a faster answer. But could you provide the >> logs and config to get a better understanding of what your issue is? >> In general, the state is maintained even in cases where a TaskManager >> fails. >> >> Best, >> Matthias >> >> On Thu, Apr 22

MemoryStateBackend Issue

Hi I see MemoryStateBackend being used in TM Log org.apache.flink.streaming.runtime.tasks.StreamTask - No state backend has been configured, using default (Memory / JobManager) MemoryStateBackend (data in heap memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null', asynchron

Re: Failed to transfer file from TaskExecutor : Vanilla Flink Cluster

the > range. > > > Best, > Yang > > Milind Vaidya 于2020年2月7日周五 上午7:03写道： > >> I figured out that it was problem with the ports. 39493/34094 were not >> accessible. So to get this working I opened all the ports 0-65535 for the >> security group. >> >

Re: Failed to transfer file from TaskExecutor : Vanilla Flink Cluster

I did try and set this to certain port range, but did not work. Thanks Milind On Wed, Feb 5, 2020 at 11:22 AM Milind Vaidya wrote: > > > > The cluster is set up on AWS with 1 Job manager and 2 task managers. > They all belong to same security group with 6123, 8081, 50100 - 50

Re: Failed to transfer file from TaskExecutor : Vanilla Flink Cluster

n you validate that all the machines can talk to each other > on the ports used by Flink (6123, 8081, ...) > If that doesn't help: > - How is the network set up? > - Are you running physical machines / VMs / containers? > - Is there a firewall involved? > > Best, > Ro

Failed to transfer file from TaskExecutor : Vanilla Flink Cluster

Hi I am trying to build a cluster for flink with 1 master and 2 workers. The program is working fine locally. The messages are read from Kafka and just printed on STDOUT. The cluster is successfully created and UI is also shows all config. But the job fails to execute on the cluster. Here are fe