+1 for the FLIP. Another side thought is that in my experience, when users want to make Flink TM use a specific hadoop/hive configuration, an easier way is to ship the corresponding conf dir and set the env variable via containerized.taskmanager.env.HADOOP_CONF_DIR.
Best, Biao Geng Archit Goyal <argo...@linkedin.com.invalid> 于2023年9月13日周三 08:00写道： > Hi All, > > If there are no further concerns about this FLIP, I will start a vote > thread tomorrow. > > Thanks, > Archit Goyal > > > From: Venkatakrishnan Sowrirajan <vsowr...@asu.edu> > Date: Thursday, August 24, 2023 at 10:21 PM > To: dev@flink.apache.org <dev@flink.apache.org> > Subject: Re: [Discuss] FLIP-355: Add parent dir of files to classpath > using yarn.provided.lib.dirs > Thanks Yang Wang. > > In that case, whenever you get a chance could you please review the PR? > > > On Wed, Aug 23, 2023, 8:15 PM Yang Wang <wangyang0...@apache.org> wrote: > > > +1 for this FLIP > > > > Maybe a FLIP is an overkill for this enhancement. > > > > > > Best, > > Yang > > > > Venkatakrishnan Sowrirajan <vsowr...@asu.edu> 于2023年8月23日周三 01:44写道： > > > > > Thanks for the FLIP, Archit. > > > > > > This is definitely quite a useful addition to the > > *yarn.provided.lib.dirs* > > > . +1. > > > > > > IMO, except for the fact that *yarn.provided.lib.dirs* (platform > specific > > > jars can be cached) takes only directories whereas *yarn.ship-files* > > (user > > > files) takes both files and dirs, the overall logic in terms of > > > constructing the classpath in both the cases should be roughly the > same. > > > > > > Referencing the PR ( > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fgithub.com%2Fapache%2Fflink%2Fpull%2F23164__%3B!!IKRxdwAv5BmarQ!cgTpodngoQAdd-qu3CvbQeAwENiu1nf0eahTH-v1NhUsSn4Y7M7sVGQYSnBjB2XgaOlyzGe7XEiU3-cAOoy84Kw%24&data=05%7C01%7Cargoyal%40linkedin.com%7C1e626c31ecaf408575b008dba52b1c6a%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638285376817262437%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2F8QiBnfVJlLZ9atF1WamCsMbAaK0TzEgcCvmd85uSnk%3D&reserved=0 > < > https://urldefense.com/v3/__https://github.com/apache/flink/pull/23164__;!!IKRxdwAv5BmarQ!cgTpodngoQAdd-qu3CvbQeAwENiu1nf0eahTH-v1NhUsSn4Y7M7sVGQYSnBjB2XgaOlyzGe7XEiU3-cAOoy84Kw$ > > > > ) with the > > > initial implementation you created as well here. > > > > > > Regards > > > Venkata krishnan > > > > > > > > > On Tue, Aug 22, 2023 at 10:09 AM Archit Goyal > > <argo...@linkedin.com.invalid > > > > > > > wrote: > > > > > > > Hi all, > > > > > > > > Gentle ping if I can get a review on the FLIP. > > > > > > > > Thanks, > > > > Archit Goyal > > > > > > > > From: Archit Goyal <argo...@linkedin.com> > > > > Date: Thursday, August 17, 2023 at 5:51 PM > > > > To: dev@flink.apache.org <dev@flink.apache.org> > > > > Subject: [Discuss] FLIP-355: Add parent dir of files to classpath > using > > > > yarn.provided.lib.dirs > > > > Hi All, > > > > > > > > I am opening this thread to discuss the proposal to add parent > > > directories > > > > of files to classpath when using yarn.provided.lib.dirs. This is > > > documented > > > > in FLIP-355 < > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FFLINK%2FFLIP*355*3A*Add*parent*dir*of*files*to*classpath*using*yarn.provided.lib.dirs__%3BKyUrKysrKysrKys!!IKRxdwAv5BmarQ!fFlyBKWuWwYcWfOcpLflTTi36tyHPiENIUry9J0ygaZY0VURnIs0glu5yafV0w0tfSsnOb9ZxDD9Cjw2TApTekVU%24&data=05%7C01%7Cargoyal%40linkedin.com%7C1e626c31ecaf408575b008dba52b1c6a%7C72f988bf86f141af91ab2d7cd011db47%7C0%7C0%7C638285376817262437%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FfyKaJDglL1caECVWX0nX8SwhHMnejfFMLjtpo5pizo%3D&reserved=0 > < > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP*355*3A*Add*parent*dir*of*files*to*classpath*using*yarn.provided.lib.dirs__;KyUrKysrKysrKys!!IKRxdwAv5BmarQ!fFlyBKWuWwYcWfOcpLflTTi36tyHPiENIUry9J0ygaZY0VURnIs0glu5yafV0w0tfSsnOb9ZxDD9Cjw2TApTekVU$ > > > > > > >. > > > > > > > > This FLIP mentions about enhancing YARN's classpath configuration to > > > > include parent directories of files in yarn.provided.lib.dirs. > > > > > > > > Please feel free to reply to this email thread and share your > opinions. > > > > > > > > Thanks, > > > > Archit Goyal > > > > > > > > > >
