Hi community, I am trying to deploy the default WordCount stream application on minikube using the official documentation at [1]. I am using minikube v1.13.0 on Ubuntu 18.04 and Kubernetes v1.19.0 on Docker 19.03.8. I could sucessfully start 1 job manager and 3 task managers using the yaml files flink-configuration-configmap.yaml, jobmanager-service.yaml, jobmanager-rest-service.yaml, jobmanager-session-deployment.yaml, and taskmanager-session-deployment.yaml (all available on the Apendix of this link [1]).
Then I would like to start the word-count stream job available on the flink jar image [2], which I believe is available since it is built inside the default flink jar distribution. What I understood that I have to do is to create the objects based on the files jobmanager-job.yaml and taskmanager-job-deployment.yaml (also available on the link [1]). And, I think that I have to replace this line below on the object jobmanager-job.yaml (spec.template.spec.containers.name[jobmanager]): args: ["standalone-job", "--job-classname", "org.apache.flink.streaming.examples.wordcount.WordCount"] Is this correct? I am not sure if this is my entire error. I am getting the message "Could not find the provided job class (org.apache.flink.streaming.examples.wordcount.WordCount) in the user lib directory (/opt/flink/usrlib)". As far as I know "/opt/flink/usrlib" is the default directory. I am not sure if I have to change the property: path /host/path/to/job/artifacts. This is my log message of the pod error. Do you guys have any idea of what I am missing in my configuration? Thanks, Felipe $ kubectl get pods NAME READY STATUS RESTARTS AGE flink-jobmanager-ftgg9 0/1 CrashLoopBackOff 3 83s $ kubectl logs flink-jobmanager-ftgg9 Starting Job Manager sed: couldn't open temporary file /opt/flink/conf/sedA699Jt: Read-only file system sed: couldn't open temporary file /opt/flink/conf/sedvZhs0w: Read-only file system /docker-entrypoint.sh: 72: /docker-entrypoint.sh: cannot create /opt/flink/conf/flink-conf.yaml: Permission denied /docker-entrypoint.sh: 91: /docker-entrypoint.sh: cannot create /opt/flink/conf/flink-conf.yaml.tmp: Read-only file system Starting standalonejob as a console application on host flink-jobmanager-ftgg9. 2020-09-18 09:15:58,801 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -------------------------------------------------------------------------------- 2020-09-18 09:15:58,804 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Preconfiguration: 2020-09-18 09:15:58,804 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JM_RESOURCE_PARAMS extraction logs: jvm_params: -Xmx1073741824 -Xms1073741824 -XX:MaxMetaspaceSize=268435456 logs: INFO [] - Loading configuration property: jobmanager.rpc.address, flink-jobmanager INFO [] - Loading configuration property: taskmanager.numberOfTaskSlots, 4 INFO [] - Loading configuration property: blob.server.port, 6124 INFO [] - Loading configuration property: jobmanager.rpc.port, 6123 INFO [] - Loading configuration property: taskmanager.rpc.port, 6122 INFO [] - Loading configuration property: queryable-state.proxy.ports, 6125 INFO [] - Loading configuration property: jobmanager.memory.process.size, 1600m INFO [] - Loading configuration property: taskmanager.memory.process.size, 1728m INFO [] - Loading configuration property: parallelism.default, 2 INFO [] - The derived from fraction jvm overhead memory (160.000mb (167772162 bytes)) is less than its min value 192.000mb (201326592 bytes), min value will be used instead INFO [] - Final Master Memory configuration: INFO [] - Total Process Memory: 1.563gb (1677721600 bytes) INFO [] - Total Flink Memory: 1.125gb (1207959552 bytes) INFO [] - JVM Heap: 1024.000mb (1073741824 bytes) INFO [] - Off-heap: 128.000mb (134217728 bytes) INFO [] - JVM Metaspace: 256.000mb (268435456 bytes) INFO [] - JVM Overhead: 192.000mb (201326592 bytes) 2020-09-18 09:15:58,805 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -------------------------------------------------------------------------------- 2020-09-18 09:15:58,805 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Starting StandaloneApplicationClusterEntryPoint (Version: 1.11.0, Scala: 2.11, Rev:d04872d, Date:2020-06-29T16:13:14+02:00) 2020-09-18 09:15:58,805 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS current user: flink 2020-09-18 09:15:58,805 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Current Hadoop/Kerberos user: <no hadoop dependency found> 2020-09-18 09:15:58,805 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.262-b10 2020-09-18 09:15:58,806 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Maximum heap size: 989 MiBytes 2020-09-18 09:15:58,806 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JAVA_HOME: /usr/local/openjdk-8 2020-09-18 09:15:58,806 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - No Hadoop Dependency available 2020-09-18 09:15:58,806 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM Options: 2020-09-18 09:15:58,806 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -Xmx1073741824 2020-09-18 09:15:58,806 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -Xms1073741824 2020-09-18 09:15:58,807 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -XX:MaxMetaspaceSize=268435456 2020-09-18 09:15:58,807 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -Dlog.file=/opt/flink/log/flink--standalonejob-0-flink-jobmanager-ftgg9.log 2020-09-18 09:15:58,807 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties 2020-09-18 09:15:58,807 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties 2020-09-18 09:15:58,807 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml 2020-09-18 09:15:58,807 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Program Arguments: 2020-09-18 09:15:58,807 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - --configDir 2020-09-18 09:15:58,808 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - /opt/flink/conf 2020-09-18 09:15:58,808 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - --job-classname 2020-09-18 09:15:58,808 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - org.apache.flink.streaming.examples.wordcount.WordCount 2020-09-18 09:15:58,808 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Classpath: /opt/flink/lib/flink-csv-1.11.0.jar:/opt/flink/lib/flink-json-1.11.0.jar:/opt/flink/lib/flink-shaded-zookeeper-3.4.14.jar:/opt/flink/lib/flink-table-blink_2.11-1.11.0.jar:/opt/flink/lib/flink-table_2.11-1.11.0.jar:/opt/flink/lib/log4j-1.2-api-2.12.1.jar:/opt/flink/lib/log4j-api-2.12.1.jar:/opt/flink/lib/log4j-core-2.12.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.12.1.jar:/opt/flink/lib/flink-dist_2.11-1.11.0.jar::: 2020-09-18 09:15:58,808 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -------------------------------------------------------------------------------- 2020-09-18 09:15:58,809 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Registered UNIX signal handlers for [TERM, HUP, INT] 2020-09-18 09:15:58,849 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not create application program. org.apache.flink.util.FlinkException: Could not find the provided job class (org.apache.flink.streaming.examples.wordcount.WordCount) in the user lib directory (/opt/flink/usrlib). at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getJobClassNameOrScanClassPath(ClassPathPackagedProgramRetriever.java:140) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.client.deployment.application.ClassPathPackagedProgramRetriever.getPackagedProgram(ClassPathPackagedProgramRetriever.java:123) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.getPackagedProgram(StandaloneApplicationClusterEntryPoint.java:110) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:78) [flink-dist_2.11-1.11.0.jar:1.11.0] [1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/kubernetes.html [2] https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java -- -- Felipe Gutierrez -- skype: felipe.o.gutierrez -- https://felipeogutierrez.blogspot.com
