Re: Branch 2.4 is cut

Ryan's proposal makes a lot of sense. Its better not to release half-baked changes in 2.4 which not only breaks a lot of the APIs released in 2.3, but also expected to change further due redesigns before 3.0 so don't see much value releasing it in 2.4. On Sun, 9 Sep 2018 at 22:42, Wenchen Fan wro

Re: Branch 2.4 is cut

Strictly speaking, data source v2 is always half-finished until we mark it as stable. We need some small milestones to move forward step by step. The redesign also happens in an incremental way. SPARK-24882 mostly focus on the "RDD" part of the API: the separation of reader factory and input parti

Re: Branch 2.4 is cut

Wenchen, can you hold off on the first RC? The half-finished changes from the redesign of the DataSourceV2 API are in master, added in SPARK-24882 , and are now in the 2.4 branch. We've had a lot of good discussion since that PR was merged to update and