Ok, so it sounds less like a "library" dependency and more like the relationship between C++ specification and compiler or between browser features and browsers. Does an implementation have to implement the 1.0 spec before implementing features in the 1.1 spec? I would assume no. Given that we may want to reword this paragraph from the docs:
> Starting with version 1.0.0, Apache Arrow utilizes two versions to describe > each release of the project: > the Format Version and the Library Version. Each Library Version has a > corresponding Format Version, > and multiple versions of the library may have the same format version. For > example, library versions > 2.0.0 and 3.0.0 may both track format version 1.0.0. There is no such thing as "tracking a version" and it doesn't make sense to say "Each library version has a corresponding Format Version" given that a library version can support and implement some pieces of every spec available. Also, I suppose it means that this JIRA (https://issues.apache.org/jira/browse/ARROW-12594) doesn't really make sense. On Wed, Apr 28, 2021 at 7:02 PM Micah Kornfield <emkornfi...@gmail.com> wrote: > > > > > A way to alleviate this is to change our status table from "pass / not > > pass" to a "since version X.Y". This would offer the transparency into > > which version it was implemented since, and would also align well with the > > idea that Rust X.Y.Z is bundled as part of the arrow release. > > > Great idea we should make this change (at least reset all check marks to > 4.0). > > > > This does not address backward incompatible changes of the format, which is > > a whole different beast (e.g. do we require all implementations to change > > prior to releasing the new version of the format?) > > > When versioning (more specifically release 1.0 of the spec) came up, we > were hoping (and I think most people) would be opposed to breaking changes > at this point. So this is a possibility, but I'd rather cross this road > when we come to it. > > -Micah > > On Wed, Apr 28, 2021 at 9:56 PM Jorge Cardoso Leitão < > jorgecarlei...@gmail.com> wrote: > > > Hi, > > > > Thanks for bringing this up. > > > > AFAIK, this has always been an issue because we do not require all > > implementations to support a type prior to releasing that new type in the > > spec. Concrete example: map has been released for a while, but Rust does > > not support it yet. Same for interval, where we still do not test in > > integration. > > > > My understanding is that once we accept that we can bump the spec without > > having all implementations implement it, people will need to consult *some > > table*. > > > > A related issue is the *some table* bit: someone using pyarrow==2 can't > > tell whether they can send type X to Rust v2 via IPC. If they go to the > > latest table [1], they can only see the latest status (4.0 in this case). > > AFAIK they would need to download the source, go to the table in .rst for > > that release, and check whether type X is supported by Rust on that > > release. > > > > A way to alleviate this is to change our status table from "pass / not > > pass" to a "since version X.Y". This would offer the transparency into > > which version it was implemented since, and would also align well with the > > idea that Rust X.Y.Z is bundled as part of the arrow release. > > > > This does not address backward incompatible changes of the format, which is > > a whole different beast (e.g. do we require all implementations to change > > prior to releasing the new version of the format?) > > > > [1] https://arrow.apache.org/docs/status.html > > > > > > > > On Thu, Apr 29, 2021 at 2:27 AM Weston Pace <weston.p...@gmail.com> wrote: > > > > > We now have independent releases. There has been some discussion (not > > > sure if it was formalized) around aligning major release versions > > > across the languages. > > > > > > There is also a potential format change coming up (new interval type). > > > > > > I think this brings up a few questions... > > > > > > Can an arrow library adopt a new format version in a minor release? > > > Semantic versioning would imply yes but then I think we could get in a > > > situation where Arrow-C++ X.Y.Z Might not be able to communicate via > > > flight using Arrow-C++ X.A.B. > > > > > > In addition, is there value in aligning format adoption across languages? > > > > > > For example, if Rust adopts format version 1.1 in version 5 and > > > pyarrow does not then users will need to consult a table to figure out > > > which versions are interoperable. > > > > > > -Weston > > > > >
