One more thing: Spark 3.0.0 has several changes regarding to DataSource V2, it would be better to evaluate the changes and do the design by also considering 3.0 changes
This actually goes the other way. We’ve been influencing the design of DataSourceV2 based on what we need for Iceberg. I’m tracking DSv2 very closely and I don’t expect any surprises. On Thu, Aug 8, 2019 at 3:36 PM Ryan Blue <rb...@netflix.com> wrote: > I think it's a great idea to branch and get ready for Spark 3.0.0. Right > now, I'm focused on getting a release out, but I can review patches for > Spark 3.0. > > Anyone know if there are nightly builds of Spark 3.0 to test with? > > On Wed, Aug 7, 2019 at 7:34 PM Saisai Shao <sai.sai.s...@gmail.com> wrote: > >> IMHO I agree that we should have a branch to track the changes for Spark >> 3.0.0. Spark 3.0.0 has several changes regarding to DataSource V2, it would >> be better to evaluate the changes and do the design by also considering 3.0 >> changes. >> >> My two cents :) >> >> Best regards, >> Saisai >> >> Edgar Rodriguez <edgar.rodrig...@airbnb.com> 于2019年8月8日周四 上午4:58写道： >> >>> Hi everyone, >>> >>> I was wondering if there's a branch tracking the changes happening in >>> Spark 3.0.0 for Iceberg. The DataSource V2 API has substantially changed >>> from the one implemented in Iceberg master branch and since Spark 3.0.0 >>> would allow us to introduce Spark SQL support then it seems interesting to >>> start tracking those changes to start evaluating some of the support as it >>> evolves. >>> >>> Thanks. >>> >>> Cheers, >>> -- >>> Edgar Rodriguez >>> >> > > -- > Ryan Blue > Software Engineer > Netflix > -- Ryan Blue Software Engineer Netflix
