Re: [SPARK-30296][SQL] Add Dataset diffing feature

2020-01-07 Thread Reynold Xin
Can this perhaps exist as an utility function outside Spark? On Tue, Jan 07, 2020 at 12:18 AM, Enrico Minack < m...@enrico.minack.dev > wrote: > > > > Hi Devs, > > > > I'd like to get your thoughts on this Dataset feature proposal. Comparing > datasets is a central operation when regressio

[SPARK-30296][SQL] Add Dataset diffing feature

2020-01-07 Thread Enrico Minack
Hi Devs, I'd like to get your thoughts on this Dataset feature proposal. Comparing datasets is a central operation when regression testing your code changes. It would be super useful if Spark's Datasets provide this transformation natively. https://github.com/apache/spark/pull/26936 Regar