Re: how to initialize few things at task managers

2022-04-20 Thread huweihua
yarn.ship-files only works in yarn environment. Maybe you could use a custom Docker entry point[1] like Austin said. [1]https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/docker/#further-customization > 2022年4月20日 上午1:19,Great Info 写道: > > I a

Re: how to initialize few things at task managers

2022-04-19 Thread Great Info
I am deploying as a docker on our servers, due to some restrictions I can only pass Keystore URLs. one option is yarn.ship-files !. can you help me with pointing to the sample code, and how job manager can ship this file? download as part of job's main function and send to all task managers.. wi

Re: how to initialize few things at task managers

2022-04-18 Thread Austin Cawley-Edwards
If you are using Kubernetes to deploy Flink, you could think about an initContainer on the TMs or a custom Docker entry point that does this initialization. Best, Austin On Mon, Apr 18, 2022 at 7:49 AM huweihua wrote: > Hi, Init stuff when task manager comes up is not an option. > But if the Ke

Re: how to initialize few things at task managers

2022-04-18 Thread huweihua
Hi, Init stuff when task manager comes up is not an option. But if the Keystore file is not changeable and you are using yarn mode, maybe you can use ‘yarn.ship-files’[1] to localize it. [1]https://nightlies.apache.org/flink/flink-docs-master/zh/docs/deployment/config/#yarn-ship-files > 2022年4月1

how to initialize few things at task managers

2022-04-16 Thread Great Info
I need to download Keystore and use it while creating the source connector, currently, I am overriding the open method