Hi Gyula, According to the log, we can see that you downloaded the source package of pemja, not the wheel package of pemja[1]. I guess you are using the m1 machine. If you install pemja from the source package, you need to have JDK, gcc tools and CPython with Numpy in the environment. I believe this can be solved after you prepared those tools, but other dependencies of pyflink 1.15 do not support m1, which makes PyFlink 1.15 unable to install and use in m1.
We have supported m1 in release 1.16[2]. If a large number of m1 users have big demand for PyFlink 1.15, I think we need to consider whether it is necessary to backport this support to 1.15, which will break our compatibility issues between minor versions. Best, Xingbo [1] https://pypi.org/project/pemja/0.1.4/ [2] https://issues.apache.org/jira/browse/FLINK-25188 Gyula Fóra <gyula.f...@gmail.com> 于2022年7月6日周三 13:56写道： > Here it is, copied from the docs essentially: > > FROM flink:1.15.0 > > > # install python3: it has updated Python to 3.9 in Debian 11 and so > install Python 3.7 from source > # it currently only supports Python 3.6, 3.7 and 3.8 in PyFlink officially. > > RUN apt-get update -y && \ > apt-get install -y build-essential libssl-dev zlib1g-dev libbz2-dev > libffi-dev git && \ > wget https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tgz && \ > tar -xvf Python-3.7.9.tgz && \ > cd Python-3.7.9 && \ > ./configure --without-tests --enable-shared && \ > make -j6 && \ > make install && \ > ldconfig /usr/local/lib && \ > cd .. && rm -f Python-3.7.9.tgz && rm -rf Python-3.7.9 && \ > ln -s /usr/local/bin/python3 /usr/local/bin/python && \ > apt-get clean && \ > rm -rf /var/lib/apt/lists/* > > # install PyFlink > RUN pip3 install apache-flink==1.15.0 > > And I am running: > docker build --tag pyflink:latest . > > This gives the following error: > > > *#6 64.12 cwd: /tmp/pip-install-9_farey_/pemja/#6 64.12 > Complete output (1 lines):#6 64.12 Include folder should be at > '/usr/local/openjdk-11/include' but doesn't exist. Please check you've > installed the JDK properly.* > > A side note: > The Dockerfile in the docs is missing git so initially I got the following > error: > > *#7 57.73 raise OSError("%r was not found" % name)#7 57.73 > OSError: 'git' was not found * > > @Weihua Hu <huweihua....@gmail.com> can you please send your working > Dockerfile? > > Gyula > > On Wed, Jul 6, 2022 at 4:16 AM Weihua Hu <huweihua....@gmail.com> wrote: > >> Hi Gyula, >> >> I can build pyFlink image successfully by following this guide. Did you >> add a dependency outside of the documentation? And could you provide your >> Dockerfile >> >> >> https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/resource-providers/standalone/docker/#using-flink-python-on-docker >> >> Best, >> Weihua >> >> >> On Tue, Jul 5, 2022 at 11:40 PM Gyula Fóra <gyula.f...@gmail.com> wrote: >> >>> Well in any case either the official image is incorrect (maybe we should >>> include JDK by default not JRE) or we should update the >>> documentation regarding the python docker build because it simply doesn't >>> work at the moment. >>> >>> I am still looking for a full working example that adds the required >>> Python packages on top of a Flink 1.15.0 base image :) >>> >>> Gyula >>> >>> On Tue, Jul 5, 2022 at 5:36 PM Weihua Hu <huweihua....@gmail.com> wrote: >>> >>>> In addition, you can try providing the Dockerfile >>>> >>>> Best, >>>> Weihua >>>> >>>> >>>> On Tue, Jul 5, 2022 at 11:24 PM Weihua Hu <huweihua....@gmail.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> The base image flink:1.15.0 is built from openjdk:11-jre, and this >>>>> image only installs jre but not jdk. >>>>> It looks like the package you want to install (pemja) depends on jdk. >>>>> you need install openjdk-11-jdk in dockerfile, >>>>> take a look to how it is installed in the official image: >>>>> >>>>> >>>>> https://hub.docker.com/layers/openjdk/library/openjdk/11-jdk/images/sha256-bc0af19c7c4f492fe6ff0c1d1c8c0e5dd90ab801385b220347bb91dbe2b4094f?context=explore >>>>> >>>>> >>>>> Best, >>>>> Weihua >>>>> >>>>> >>>>> On Tue, Jul 5, 2022 at 3:50 PM Gyula Fóra <gyula.f...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi All! >>>>>> >>>>>> I have been trying to experiment with the Flink python support on >>>>>> Kubernetes but I got stuck creating a custom image with all the necessary >>>>>> python libraries. >>>>>> >>>>>> I found this guide in the docs: >>>>>> https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/deployment/resource-providers/standalone/docker/#using-flink-python-on-docker >>>>>> >>>>>> However when I try to build a custom image using it, I get the >>>>>> following error: >>>>>> >>>>>> #7 131.7 Collecting pemja==0.1.4 >>>>>> #7 131.8 Downloading pemja-0.1.4.tar.gz (32 kB) >>>>>> #7 131.9 ERROR: Command errored out with exit status 255: >>>>>> #7 131.9 command: /usr/local/bin/python3.7 -c 'import sys, >>>>>> setuptools, tokenize; sys.argv[0] = >>>>>> '"'"'/tmp/pip-install-y6o6djs1/pemja/setup.py'"'"'; >>>>>> __file__='"'"'/tmp/pip-install-y6o6djs1/pemja/setup.py'"'"';f=getattr(tokenize, >>>>>> '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', >>>>>> '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' >>>>>> egg_info --egg-base /tmp/pip-pip-egg-info-47iid6nm >>>>>> #7 131.9 cwd: /tmp/pip-install-y6o6djs1/pemja/ >>>>>> #7 131.9 Complete output (1 lines): >>>>>> #7 131.9 Include folder should be at >>>>>> '/usr/local/openjdk-11/include' but doesn't exist. Please check you've >>>>>> installed the JDK properly. >>>>>> >>>>>> Has anyone seen this before and knows the solution? >>>>>> >>>>>> Thanks >>>>>> Gyula >>>>>> >>>>>
