Hi Yik San, 1) what `--pyFiles` is used for: All the files specified via `--pyFiles` will be put in the PYTHONPATH of the Python worker during execution and then they will be available for the Python user-defined functions during execution.
2) validate for the files passed to `--pyFiles` Currently it will not validate the files passed to this argument. I also think that it’s not necessary and not able to perform such kind of check. Do you have any advice for this? Regards, Dian > 2021年4月26日 下午8:45，Yik San Chan <evan.chanyik...@gmail.com> 写道： > > Hi community, > > In > https://ci.apache.org/projects/flink/flink-docs-stable/dev/python/python_config.html > > <https://ci.apache.org/projects/flink/flink-docs-stable/dev/python/python_config.html>, > regarding python.files: > > > Attach custom python files for job. > > This makes readers think only Python files are allowed here. However, in > https://ci.apache.org/projects/flink/flink-docs-stable/deployment/cli.html#submitting-pyflink-jobs > > <https://ci.apache.org/projects/flink/flink-docs-stable/deployment/cli.html#submitting-pyflink-jobs>: > > ./bin/flink run \ > --python examples/python/table/batch/word_count.py \ > --pyFiles file:///user.txt,hdfs:///$namenode_address/username.txt > It is obviously including .txt file that is not Python files. > > I believe it is contradictory here. Can anyone confirm? > > Best, > Yik San
