Example flink run with security options? Running on k8s in my case

[Adam Roberts]

Hey everyone, I've been experimenting with Flink using https://github.com/GoogleCloudPlatform/flink-on-k8s-operator and I believe I've successfully deployed a JobManager and TaskManager with security enabled, and a self-signed certificate (the pods come up great).

 

However, I can't do much with this - I can't port-forward and access the UI, nor can I submit jobs to it by running another pod and using the DNS name lookup of the service.

 

I always get

 

The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: java.util.concurrent.ExecutionException: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.

 

...

 

Caused by: org.apache.flink.runtime.rest.ConnectionClosedException: Channel became inactive.
    ... 37 more

 

 

and this is even with all of the -D security options provided.

 

The versions of Flink are the same for both my Job and my FlinkCluster (1.11.1).

 

Is this a sensible thing to do? If I weren't using the operator for example, would users be expected to flink run with all of these options?

 

Does anything look odd here? My guess is because security's on, the Job Manager refuses to talk to my submitter.

 

Running as the flink user in the container, I do

 

      securityContext:

        runAsUser: 9999

        runAsGroup: 9999

      containers:

      - name: wordcount

        image: adamroberts/mycoolflink:latest

        args:

        - /opt/flink/bin/flink

        - run

        - -D

        - security.ssl.rest.keystore=/etc/flink-secrets/flink-tls-keystore.key

        - -D 

        - security.ssl.rest.truststore=/etc/flink-secrets/flink-tls-ca-truststore.jks

        - -D 

        - security.ssl.rest.keystore-password=thepass # Replace with value of flink-tls-keystore.password

        - -D 

        - security.ssl.rest.key-password=thepass # Replace with value of tls.p12.password

        - -D 

        - security.ssl.rest.truststore-password=thepass # Replace with value of flink-tls-ca.truststore.password

        - -D 

        - security.ssl.internal.keystore=/etc/flink-secrets/flink-tls-keystore.key

        - -D 

        - security.ssl.internal.truststore=/etc/flink-secrets/flink-tls-ca-truststore.jks

        - -D 

        - security.ssl.internal.keystore-password=thepass # Replace with value of flink-tls-keystore.password

        - -D 

        - security.ssl.internal.key-password=thepass # Replace with value of flink-tls-keystore.password

        - -D 

        - security.ssl.internal.truststore-password=thepass # Replace with value of flink-tls-truststore.password

        - -m

        - tls-flink-cluster-1-11-jobmanager:8081

        - /opt/flink/examples/batch/WordCount.jar 

        - --input 

        - /opt/flink/NOTICE

 

with the secrets mounted in at the above location (if I exec into my container, I can see they're all there OK). Note that it is a read-only file system.

 

adamroberts/mycoolflink (at this time of this email) is just based on https://github.com/apache/flink-docker.

 

Thanks!

 

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU