Hi David, Sorry I missed your reply.
>>>Since you're mentioning docker, I assume you're deploying your application to k8s. Is that correct? yes, I am using k8s. >>>For handcrafted Kubernetes deployments, you can simply download the jar into the user lib folder in an init container [1]. You can usually reuse existing docker images to download the jar. For example, for S3, the AWS CLI image will do the trick [2]. I believe that since Flink already supports different filesystems, we can utilize the same code without depending on any other CLI. This would help us save the time currently taken by the init container. Not only can application mode clusters on Kubernetes utilize this, but other clusters can also use it to download additional plugins before starting the cluster. Thanks Surendra On Sat, Mar 4, 2023 at 12:41 AM Surendra Singh Lilhore < surendralilh...@apache.org> wrote: > Hi Team, > > > > According to the Flink documentation, in the APP mode, the application jar > should be bundled with the Flink image. However, building an image for each > new application can be difficult. Can we introduce new commands that will > help to download the required jar locally before starting Flink JM or TM > containers? This should be a simple command that depends on the supported > file system (S3, HDFS, ABFS) in Flink, and the command format should be > something like this: > > *./flink fs-download <Remote-FS Path> <Local-Path>* > > Example: > > *./flink fs-download > abfs://mycontai...@storageaccount.dfs.core.windows.net/jars /tmp/appjars > <http://mycontai...@storageaccount.dfs.core.windows.net/jars%0D/tmp/appjars>* > > I have already tested this in my cluster, and it is working fine. Before > raising a JIRA ticket, I would like to get suggestions from the community. > > > Thanks and Regards > Surendra >
