Good questions. Is there a way to get this work vetted somewhere outside the project to start with, say in a fork or a separate project . . . ?
This is essentially what I’m suggesting by not releasing it with the JVM version. Let’s get the project bootstrapped and have a place to work on it, and then decide when we think it is ready for release later. I don’t want to keep it in a PR where it is hard to build the python community and collaborate on it. Do you think this needs to be a separate repo, or can we just have separate releases? Would every feature added to the Java version need to be mirrored in Python? I think that the spec should be used to coordinate across implementations, but that those implementations can have different features and degrees of support. It would be fine if python didn’t have support for the encryption structures until someone needs that support from Python and adds it. Otherwise, we’re asking too much of contributors: go fix this in another language that you may not know or be comfortable in. On Thu, Feb 28, 2019 at 1:36 PM Matt Cheah <mch...@palantir.com> wrote: > Thanks Ted for the contribution! I agree that this is important to have. > > > > Is there a way to get this work vetted somewhere outside the project to > start with, say in a fork or a separate project, and then to incorporate it > when users have deployed in production and can provide concrete positive > feedback? This is a lot of code to be added at once. > > > > I was also wondering how the Python implementation would track changes to > the Java implementation. For example, we have since added support for > encryption and custom file I/O since this PR was first opened. How would > Python take that change? Would every feature added to the Java version need > to be mirrored in Python? If so, should the mirroring always happen in-line > with the PRs that make the change in Java, or can feature parity be delayed? > > > > -Matt Cheah > > > > *From: *Ryan Blue <rb...@netflix.com.INVALID> > *Reply-To: *"dev@iceberg.apache.org" <dev@iceberg.apache.org>, " > rb...@netflix.com" <rb...@netflix.com> > *Date: *Thursday, February 28, 2019 at 12:27 PM > *To: *Iceberg Dev List <dev@iceberg.apache.org> > *Subject: *[DISCUSS] Python implementation > > > > Hi everyone, > > > > One of our contributors, Ted, has done a lot of work on an initial python > implementation and Uwe was kind enough to review it. Here's the PR: > > > > https://github.com/apache/incubator-iceberg/pull/54 [github.com] > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_incubator-2Diceberg_pull_54&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=hzwIMNQ9E99EMYGuqHI0kXhVbvX3nU3OSDadUnJxjAs&m=AQgareLPlJM39OyX28V7S62lnFTt8JBbpkGKJFIXfG8&s=ZsxKBdEvEjfxb33B8hKgoMRxaXQPKz65LJ1YdhUnoRg&e=> > > > > Because this is a brand-new implementation, the PR is huge: 157 new files. > That makes it really tough to review in depth, and also really time > consuming to update and maintain. What I suggest is committing the PR as-is > now that it has passed a round of reviews. Then we can improve it in > smaller pull requests. > > > > Are there any objections to this plan or other thoughts? > > > > I think that the python implementation would not be included in the first > Apache Iceberg release. I would prefer to release the python implementation > on a separate release cycle so that Java blockers don't prevent a Python > bug fix and vice versa. > > > > rb > > > > -- > > Ryan Blue > > Software Engineer > > Netflix > -- Ryan Blue Software Engineer Netflix
