> Also, unrelated, is there a schedule for the sync calls? Will try and carve out some free time for the next one :)
It is every other Wednesday at noon EST. Here is the original announcement with more details: https://lists.apache.org/thread.html/raa72e1a8a3ad5dbb8366e9609a041eccca87f85545c3bc3d85170cfc%40%3Cdev.arrow.apache.org%3E On Sun, Feb 14, 2021 at 8:29 AM Ruan Pearce-Authers <r...@reservoirdb.com> wrote: > I'd be interested in helping spec this out, it's especially tricky atm to > track down issues when integrating DataFusion into the same binary as other > medium/large dependencies. > > Recently hit a really specific issue where DataFusion depends on Parquet, > which supports various compression algs, including Brotli, and actix-web > also depends on a slightly different Rust implementation of Brotli. Both of > these Brotli libs package the same underlying C lib separately, resulting > in multiply-defined symbols compiling using msvc (and maybe on other > platforms? didn't test in CI in the end). > > Got a quick interim hack [1] in place for my use case which doesn't really > use Parquet, so it's not pressing, but would be awesome to sort this > properly upstream. > > I guess the only major tradeoff of having a comprehensive feature setup is > that it could make testing slightly harder, in terms of making sure no-one > breaks the build for specific feature combinations; this can always be > mitigated with more CI though (yay, unlimited Actions minutes for public > repos). > > Also, unrelated, is there a schedule for the sync calls? Will try and > carve out some free time for the next one :) > > [1] > https://github.com/reservoirdb/arrow/commit/e63e157927a552ecf1a6f63ec401f0b6157b5468 > > -----Original Message----- > From: Andrew Lamb <al...@influxdata.com> > Sent: 14 February 2021 11:14 > To: dev <dev@arrow.apache.org> > Subject: [Rust] [DataFusion] Topic for next Rust Sync Call > > I would like to add the following item to the agenda call for the next > Rust sync call: > > Dependencies > > Background: As the dependency stack gets larger, it will be harder to use > DataFusion as an embedded query engine and the compile / dev times will get > higher. > > As we expand the supported functions of DataFusion this problem is likely > to get worse. For example > https://github.com/apache/arrow/pull/9243#discussion_r575716759 and > https://github.com/apache/arrow/pull/9139 > > Proposal: Add Rust "features" to the datafusion crate and make many of the > new dependencies optional (so that we had features like regex and unicode > and hash which would only pull in the dependencies / have those functions > if the features were enabled.) This approach has worked well for Arrow > (which has only chrono and num as required dependencies) >
