Hi Xingbo,
Thanks a lot for bringing up this discussion. Actually the size limit already becomes an issue during releasing 1.11.3 and 1.12.1. It blocks us to publish PyFlink packages to PyPI during the release as there is no enough space left (PS: already published the packages after increasing the size limit). Considering that the total package size are about 1.5GB (220MB * 7) for each release, it makes sense to split the PyFlink package. It could reduce the total package size to about 250MB (3MB * 7 + 220 MB) for each release. We don’t need to increase the size limit any more in the next few years as currently we still have about 7.5 GB space left. So +1 from my side. Regards, Dian > 2021年3月12日 下午2:30，Xingbo Huang <hxbks...@gmail.com> 写道： > > Hi everyone, > > Since release-1.11, pyflink has introduced cython support and we will > release 7 packages (for different platforms and Python versions) to PyPI > for each release and the size of each package is more than 200MB as we need > to bundle the jar files into the package. The entire project space in PyPI > grows very fast, and we need to apply to PyPI for more project space > frequently. Please refer to [https://github.com/pypa/pypi-support/issues/831] > for more details. > > The root cause to this problem is that we bundled the jar files in each > package. This is actually unnecessary if we could extract the jar files > into a separate package which is dedicated to hold the jar files. > > I’d like to propose to split the pyflink package into two packages: the > original apache-flink and apache-flink-libraries (Any suggestions for the > name?). The package apache-flink-libraries only contains jar files and > there is only one apache-flink-libraries package for each release. The > package apache-flink depends on apache-flink-libraries and for users, they > still only need to install apache-flink and there is nothing different from > before. We still need to release multiple wheel packages of apache-flink. > However, the size will be very small as it doesn't contain the jar files > any more. > > Looking forward to your feedback. > > Best, > > Xingbo
