Hi everyone, Since release-1.11, pyflink has introduced cython support and we will release 7 packages (for different platforms and Python versions) to PyPI for each release and the size of each package is more than 200MB as we need to bundle the jar files into the package. The entire project space in PyPI grows very fast, and we need to apply to PyPI for more project space frequently. Please refer to [https://github.com/pypa/pypi-support/issues/831] for more details.
The root cause to this problem is that we bundled the jar files in each package. This is actually unnecessary if we could extract the jar files into a separate package which is dedicated to hold the jar files. I’d like to propose to split the pyflink package into two packages: the original apache-flink and apache-flink-libraries (Any suggestions for the name?). The package apache-flink-libraries only contains jar files and there is only one apache-flink-libraries package for each release. The package apache-flink depends on apache-flink-libraries and for users, they still only need to install apache-flink and there is nothing different from before. We still need to release multiple wheel packages of apache-flink. However, the size will be very small as it doesn't contain the jar files any more. Looking forward to your feedback. Best, Xingbo
