explain it and you'll know what happens under the covers. i.e. Use explain on the Dataset.
Jacek On 25 Apr 2017 12:46 a.m., "Lavelle, Shawn" <shawn.lave...@osii.com> wrote: > Hello Spark Users! > > Does the Spark Optimization engine reduce overlapping column ranges? > If so, should it push this down to a Data Source? > > Example, > > This: Select * from table where col between 3 and 7 OR col between 5 > and 9 > > Reduces to: Select * from table where col between 3 and 9 > > > > > > Thanks for your insight! > > > ~ Shawn M Lavelle > > > > > > > Shawn Lavelle > Software Development > > 4101 Arrowhead Drive > Medina, Minnesota 55340-9457 > Phone: 763 551 0559 <(763)%20551-0559> > Fax: 763 551 0750 <(763)%20551-0750> > *Email:* shawn.lave...@osii.com > *Website: **www.osii.com* <http://www.osii.com> >