This seems reasonable. The flexibility with CI is a positive for sure. > 1. Cargo doesn't allow cyclic dependency. So if the parquet sub-crate > depends on arrow, we can't reference parquet in arrow.
This is my only concern, the Rust implementation is evolving rapidly and adopting workspaces may reduce our flexibility, I can’t think of a specific situation right now but might this be a problem in the future? If we have this restriction we have to agree on which way dependencies flow, what is your preference? I have not used workspaces in anger but it seems it is designed for crates to flow up? i.e. that arrow would be allowed to reference the parquet sub-crate. P From: Renjie Liu<mailto:liurenjie2...@gmail.com> Sent: Thursday, December 27, 2018 10:09 PM To: dev@arrow.apache.org<mailto:dev@arrow.apache.org> Subject: Re: [Rust] move parquet into a separate sub-crate Cool. It may also be worthy to put adapters into a separate crate. On Fri, Dec 28, 2018 at 4:10 AM Chao Sun <sunc...@apache.org> wrote: > Hi, > > It just occurs to me that it may be a better idea to move the parquet > module into a separate sub-crate by using cargo workspaces > <https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html>. The > advantage is that we can make the project more modular (in future, we may > want to add more sub-crates such as arrow/parquet_derive, orc, gandiva, > etc), and allow us to run CI jobs separately on each crate. > > Some small caveats: > 1. Cargo doesn't allow cyclic dependency. So if the parquet sub-crate > depends on arrow, we can't reference parquet in arrow. This doesn't seem > like an issue though since arrow itself should be physical on-disk format > independent. I also didn't see any reference on parquet in cpp/src/arrow. > 2. The path dependency used in workspace has to be changed to a version > number when we do "cargo publish". This should be added to the release > instructions and committer who performs the job should do the extra step. > > Thoughts? > > Chao > -- Renjie Liu Software Engineer, MVAD
