Dear all, As some of you know, there is a standard in Python called manylinux ( https://www.python.org/dev/peps/pep-0513/) to package binary executables and libraries into a “wheel” in a way that allows the code to be run on a wide variety of Linux distributions. This is very convenient for Python users, since such libraries can be easily installed via pip.
This standard is also important for a second reason: If many different wheels are used together in a single Python process, adhering to manylinux ensures that these libraries work together well and don’t trip on each other’s toes (this could easily happen if different versions of libstdc++ are used for example). Therefore *even if support for only a single distribution like Ubuntu is desired*, it is important to be manylinux compatible to make sure everybody’s wheels work together well. TensorFlow and PyTorch unfortunately don’t produce manylinux compatible wheels. The challenge is due, at least in part, to the need to use nvidia-docker to build GPU binaries [10]. This causes various levels of pain for the rest of the Python community, see for example [1] [2] [3] [4] [5] [6] [7] [8]. The purpose of the e-mail is to get a discussion started on how we can make TensorFlow and PyTorch manylinux compliant. There is a new standard in the works [9] so hopefully we can discuss what would be necessary to make sure TensorFlow and PyTorch can adhere to this standard in the future. It would make everybody’s lives just a little bit better! Any ideas are appreciated. @soumith: Could you cc the relevant list? I couldn't find a pytorch dev mailing list. Best, Philipp. [1] https://github.com/tensorflow/tensorflow/issues/5033 [2] https://github.com/tensorflow/tensorflow/issues/8802 [3] https://github.com/primitiv/primitiv-python/issues/28 [4] https://github.com/zarr-developers/numcodecs/issues/70 [5] https://github.com/apache/arrow/pull/3177 [6] https://github.com/tensorflow/tensorflow/issues/13615 [7] https://github.com/pytorch/pytorch/issues/8358 [8] https://github.com/ray-project/ray/issues/2159 [9] https://www.python.org/dev/peps/pep-0571/ [10] https://github.com/tensorflow/tensorflow/issues/8802#issuecomment-291935940
