Hi Wes, others, A few thoughts from a user. Firstly, I completely understand your frustration. I myself have delved into a bit of packaging for many scientific computing packages, like ROOT from CERN, although not at the scale of users that you face here.
AIU, wheels are a Python-first spec, whereas Arrow is a C++ first library, with python bindings. I feel this is what causes the friction in the build chain for wheels. That said, I would like to propose the following. On Mon, Jul 15, 2019 at 10:06:41PM -0500, Wes McKinney wrote: > > * Our wheel become much more complex due to Flight (requiring gRPC, > OpenSSL, and other dependencies) and Gandiva (requiring LLVM and more) Disable the more advanced features and release reduced feature set wheels, say, only with: 1. core data structures, Table, etc, 2. various serialisation support (parquet, orc, etc), and 3. plasma. My justification being, it covers a significant proportion of the relatively non-expert usecases. (1) covers the interaction with other Python libraries, particularly pandas, (2) covers most I/O requirements, and plasma along with providing a way to manage Arrow objects in-memory for more advanced architectures, it also serves as a relatively simple bridge to other languages. Any users requiring Gandiva or Flight on Python could easily "upgrade" to the conda-forge releases. What do you think? Cheers, -- Suvayu Open source is the future. It sets us free.
