I think what you suggest is highly dependent on who does the work. The first question is who is willing to do the work. Given that they are volunteers, they'd probably need to propose something like this (but with there own flavors/choices) and then we'd have to figure out how this communicated to users (especially in the context that the same package would potentially have different capabilities if used pip vs conda).
On Mon, Jul 15, 2019 at 8:52 PM Suvayu Ali <fatkasuvayu+li...@gmail.com> wrote: > Hi Wes, others, > > A few thoughts from a user. Firstly, I completely understand your > frustration. I myself have delved into a bit of packaging for many > scientific computing packages, like ROOT from CERN, although not at the > scale of users that you face here. > > AIU, wheels are a Python-first spec, whereas Arrow is a C++ first library, > with python bindings. I feel this is what causes the friction in the build > chain for wheels. That said, I would like to propose the following. > > On Mon, Jul 15, 2019 at 10:06:41PM -0500, Wes McKinney wrote: > > > > * Our wheel become much more complex due to Flight (requiring gRPC, > > OpenSSL, and other dependencies) and Gandiva (requiring LLVM and more) > > Disable the more advanced features and release reduced feature set wheels, > say, only with: > 1. core data structures, Table, etc, > 2. various serialisation support (parquet, orc, etc), and > 3. plasma. > > My justification being, it covers a significant proportion of the > relatively non-expert usecases. (1) covers the interaction with other > Python libraries, particularly pandas, (2) covers most I/O requirements, > and plasma along with providing a way to manage Arrow objects in-memory for > more advanced architectures, it also serves as a relatively simple bridge > to other languages. Any users requiring Gandiva or Flight on Python could > easily "upgrade" to the conda-forge releases. > > What do you think? > > Cheers, > > -- > Suvayu > > Open source is the future. It sets us free. >
