Hi Chesnay, We will publish binary packages separately for: 1) Python 3.5 / 3.6 / 3.7 / 3.8 (since 1.12) separately 2) Linux / Mac separately
Besides, there is also a source package which is used when none of the above binary packages is usable, e.g. for Window users. PS: publishing multiple binary packages is very common in Python world, e.g. Beam published 22 packages in 2.28, Pandas published 16 packages in 1.2.3 [2]. We could also publishing more packages if we splitting the packages as the cost of adding another package will be very small. Regards, Dian [1] https://pypi.org/project/apache-beam/#files <https://pypi.org/project/apache-beam/#files> [2] https://pypi.org/project/pandas/#files Hi Xintong, Yes, you are right that there is 9 packages in 1.12 as we added Python 3.8 support in 1.12. Regards, Dian > 2021年3月16日 下午7:45，Xintong Song <tonysong...@gmail.com> 写道： > > And it's not only uploaded to PyPI, but the ASF mirrors as well. > > https://dist.apache.org/repos/dist/release/flink/flink-1.12.2/python/ > > Thank you~ > > Xintong Song > > > > On Tue, Mar 16, 2021 at 7:41 PM Xintong Song <tonysong...@gmail.com> wrote: > >> Actually, I think it's 9 packages, not 7. >> >> Check here for the 1.12.2 packages. >> https://pypi.org/project/apache-flink/#files >> >> Thank you~ >> >> Xintong Song >> >> >> >> On Tue, Mar 16, 2021 at 7:08 PM Chesnay Schepler <ches...@apache.org> >> wrote: >> >>> Am I reading this correctly that we publish 7 different artifacts just >>> for python? >>> What does the release matrix look like? >>> >>> On 3/16/2021 3:45 AM, Dian Fu wrote: >>>> Hi Xingbo, >>>> >>>> >>>> Thanks a lot for bringing up this discussion. Actually the size limit >>> already becomes an issue during releasing 1.11.3 and 1.12.1. It blocks us >>> to publish PyFlink packages to PyPI during the release as there is no >>> enough space left (PS: already published the packages after increasing the >>> size limit). >>>> >>>> >>>> Considering that the total package size are about 1.5GB (220MB * 7) for >>> each release, it makes sense to split the PyFlink package. It could reduce >>> the total package size to about 250MB (3MB * 7 + 220 MB) for each release. >>> We don’t need to increase the size limit any more in the next few years as >>> currently we still have about 7.5 GB space left. >>>> >>>> So +1 from my side. >>>> >>>> Regards, >>>> Dian >>>> >>>>> 2021年3月12日 下午2:30，Xingbo Huang <hxbks...@gmail.com> 写道： >>>>> >>>>> Hi everyone, >>>>> >>>>> Since release-1.11, pyflink has introduced cython support and we will >>>>> release 7 packages (for different platforms and Python versions) to >>> PyPI >>>>> for each release and the size of each package is more than 200MB as we >>> need >>>>> to bundle the jar files into the package. The entire project space in >>> PyPI >>>>> grows very fast, and we need to apply to PyPI for more project space >>>>> frequently. Please refer to [ >>> https://github.com/pypa/pypi-support/issues/831] >>>>> for more details. >>>>> >>>>> The root cause to this problem is that we bundled the jar files in each >>>>> package. This is actually unnecessary if we could extract the jar files >>>>> into a separate package which is dedicated to hold the jar files. >>>>> >>>>> I’d like to propose to split the pyflink package into two packages: the >>>>> original apache-flink and apache-flink-libraries (Any suggestions for >>> the >>>>> name?). The package apache-flink-libraries only contains jar files and >>>>> there is only one apache-flink-libraries package for each release. The >>>>> package apache-flink depends on apache-flink-libraries and for users, >>> they >>>>> still only need to install apache-flink and there is nothing different >>> from >>>>> before. We still need to release multiple wheel packages of >>> apache-flink. >>>>> However, the size will be very small as it doesn't contain the jar >>> files >>>>> any more. >>>>> >>>>> Looking forward to your feedback. >>>>> >>>>> Best, >>>>> >>>>> Xingbo >>> >>> >>>
