Thanks Andrew! It seems there is a drastic change in 3.0, going through it.
-Aakash On Tue, Dec 17, 2019 at 11:01 AM Andrew Melo <andrew.m...@gmail.com> wrote: > Hi Aakash > > On Tue, Dec 17, 2019 at 12:42 PM aakash aakash <email2aak...@gmail.com> > wrote: > >> Hi Spark dev folks, >> >> First of all kudos on this new Data Source v2, API looks simple and it >> makes easy to develop a new data source and use it. >> >> With my current work, I am trying to implement a new data source V2 >> writer with Spark 2.3 and I was wondering how I will get the info about >> partition by columns. I see that it has been passed to Data Source V1 from >> DataFrameWriter but not for V2. >> > > Not directly related to your Q, but just so you're aware, the DSv2 API > evolved from 2.3->2.4 and then again for 2.4->3.0. > > Cheers > Andrew > > >> >> >> Thanks, >> Aakash >> >
