Query on using Apache Flink with Confluent Kafka

[elakiya udhayanan]

Hi Team,

I have a requirement to read kafka events through Apache Flink and do
processing with the same.

Now this kafka topic which produces the event to the Apache Flink is a
confluent kafka and it is hosted as a kubernetes pod in the Docker
container.

The actual problem is I am unable to consume the message that is being
produced by the Kafka topic. I am attaching my pom.xml and the main class
which I used to consume the message.

Also I would like to understand if the below queries are doable using
Apache Flink.
1. Can we consume and produce messages to confluent Kafka that is either
hosted in a docker container or in a eks cluster ?
2. Can we write a spring cloud based application using Apache flink and
deploy the same as a flink job ?

Please help me to solve my issue.

Thanks,
Elakiya

KafkaDemo.java
Description: Binary data

<project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";>
  <modelVersion>4.0.0</modelVersion>
  <groupId>kafka</groupId>
  <artifactId>kafka</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  
   <properties>
    <!-- Plugins -->
    <maven-assembly-plugin.version>3.3.0</maven-assembly-plugin.version>
    <maven-compiler-plugin.version>3.8.0</maven-compiler-plugin.version>
    <!-- compile for Java 8 -->
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
    <!-- Dependencies -->
    <!-- General -->
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>
  <dependencies>
	
	<dependency>
 	 <groupId>org.apache.flink</groupId>
  		<artifactId>flink-java</artifactId>
  	<version>1.14.4</version>
  	<scope>provided</scope>
	</dependency>

	<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-streaming-java_2.11</artifactId>
  <version>1.14.4</version>
  <scope>provided</scope>
</dependency>

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients_2.11</artifactId>
  <version>1.14.4</version>
  <scope>provided</scope>
</dependency>

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-connector-kafka_2.11</artifactId>
    <version>1.14.4</version>
</dependency>


<dependency>
      <groupId>commons-logging</groupId>
      <artifactId>commons-logging</artifactId>
      <version>1.1.1</version>
    </dependency>
   

</dependencies>

<build>
	<plugins>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-assembly-plugin</artifactId>
			<version>3.1.0</version>
				<configuration>
					<!-- get all project dependencies -->
					<descriptorRefs>
						<descriptorRef>jar-with-dependencies</descriptorRef>
					</descriptorRefs>
					<!-- MainClass in mainfest make a executable jar -->
					<archive>
					  <manifest>
						<mainClass>com.kafkastream.KafkaDemo</mainClass>
					  </manifest>
					</archive>

				</configuration>
				<executions>
				  <execution>
					<id>make-assembly</id>
                                        <!-- bind to the packaging phase -->
					<phase>package</phase> 
					<goals>
						<goal>single</goal>
					</goals>
				  </execution>
				</executions>
			</plugin>
	</plugins>
</build>
</project>