Re: More Robust DataSource Parameters

2016-03-07 Thread Reynold Xin
Hi Hamel, Sorry for the slow reply. Do you mind writing down the thoughts in a document, with API sketches? I think all the devils are in the details of the API for this one. If we can design an API that is type-safe, supports all languages, and also can be stable, then it sounds like a great ide

Re: More Robust DataSource Parameters

2016-02-27 Thread Hamel Kothari
Thanks for the flags Reynold. 1. For the 4+ languages, these are just on the consumption side (i.e. you can't write a data source in Python or SQL, correct), right? ? If this is correct and you can only write data sources in the JVM languages than that makes this story a lot easier. On the DataSou

Re: More Robust DataSource Parameters

2016-02-26 Thread Reynold Xin
Thanks for the email. This sounds great in theory, but might run into two major problems: 1. Need to support 4+ programming languages (SQL, Python, Java, Scala) 2. API stability (both backward and forward) On Fri, Feb 26, 2016 at 8:44 AM, Hamel Kothari wrote: > Hi devs, > > Has there been an