>From the exception stack, I suspect that it's not true that all the cluster nodes have installed PyFlink. Could you check the node where the application master executes and check if it's true?
Regards, Dian On Wed, Apr 9, 2025 at 9:52 AM Joska H <handsome...@gmail.com> wrote: > > We have installed PyFlink on all nodes of the cluster. Each node can > successfully execute python /example/word_count.py and can also execute jobs > in yarn-per-job mode. However, when submitting jobs in application mode, an > error occurs indicating that the package cannot be found. > The submission command is: > > flink run-application --target yarn-application \ > -py /data/module/flink-1.18.1/work_space/scripts/test.py \ > -pyexec /usr/bin/python \ > -Dyarn.application.name=test_applicationmode \ > -Dpython.client.executable=/usr/bin/python > > The error message is: > > 2025-04-09 09:45:14,846 INFO org.apache.flink.client.python.PythonDriver > [] - --------------------------- Python Process Started > -------------------------- > 2025-04-09 09:45:14,858 INFO org.apache.flink.client.python.PythonDriver > [] - Traceback (most recent call last): > File "/data/module/flink-1.18.1/work_space/scripts/test.py", line 1, in > <module> > from pyflink.common import Row > ModuleNotFoundError: No module named 'pyflink' > > We hope this information helps clarify the situation. Looking forward to your > advice. > Best regards， > Shiquan > > ________________________________ > From: Dian Fu <dian0511...@gmail.com> > Sent: Wednesday, April 9, 2025 1:31 AM > To: Joska H <handsome...@gmail.com> > Cc: user@flink.apache.org <user@flink.apache.org> > Subject: Re: PyFink Application Mode Dependency Resolution > > Have you installed PyFlink in the used Python environment? > > Regards, > Dian > > On Tue, Apr 1, 2025 at 2:43 PM Joska H <handsome...@gmail.com> wrote: > > > > When submitting PyFink jobs via Application Mode: > > > > ./bin/flink run-application -t yarn-application \ > > -Djobmanager.memory.process.size=1024m \ > > -Dtaskmanager.memory.process.size=1024m \ > > -Dyarn.application.name=test_applicationmode \ > > -pyclientexec /usr/bin/python \ > > -pyexec /usr/bin/python \ > > -python.pythonpath /usr/bin/python \ > > -py /data/module/flink-1.18.1/work_space/scripts/test.py > > > > We encounter ModuleNotFoundError:pyflink despite all worker nodes having > > the required Python packages. However, these jobs work properly in YARN > > Per-Job mode. Could you give me some advise. > > > > We would appreciate any code samples or configuration references you could > > provide. Please find attached our full submission script and error logs for > > reference. > > > > Best regards, > > Shiquan Song. > > > >
