Hi Surendra, Since you're mentioning docker, I assume you're deploying your application to k8s. Is that correct?
For handcrafted Kubernetes deployments, you can simply download the jar into the user lib folder in an init container [1]. You can usually reuse existing docker images to download the jar. For example, for S3, the AWS CLI image will do the trick [2]. In general, my take is that this doesn't belong to the Flink itself (we should keep the core feature matrix dense) but to the deployment/orchestration layer around it (e.g., the AF Kubernetes Operator [3]). [1] https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ [2] https://hub.docker.com/r/amazon/aws-cli [3] https://github.com/apache/flink-kubernetes-operator Best, D. On Fri, Mar 3, 2023 at 8:11 PM Surendra Singh Lilhore < surendralilh...@apache.org> wrote: > Hi Team, > > > > According to the Flink documentation, in the APP mode, the application jar > should be bundled with the Flink image. However, building an image for each > new application can be difficult. Can we introduce new commands that will > help to download the required jar locally before starting Flink JM or TM > containers? This should be a simple command that depends on the supported > file system (S3, HDFS, ABFS) in Flink, and the command format should be > something like this: > > *./flink fs-download <Remote-FS Path> <Local-Path>* > > Example: > > *./flink fs-download > abfs://mycontai...@storageaccount.dfs.core.windows.net/jars /tmp/appjars > <http://mycontai...@storageaccount.dfs.core.windows.net/jars > /tmp/appjars>* > > I have already tested this in my cluster, and it is working fine. Before > raising a JIRA ticket, I would like to get suggestions from the community. > > > Thanks and Regards > Surendra >
