Re: This works to filter transactions older than certain months

Forgot to mention Spark 1.6.1 Hive 2.0 Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com On 28 March 2016

Re: This works to filter transactions older than certain months

Snippet. import org.apache.spark.sql.functions._ import java.sql.{Date, Timestamp} val HiveContext = new org.apache.spark.sql.hive.HiveContext(sc) val df = sqlContext.read.format("com.databricks.spark.csv").option("inferSchema", "true").option("header", "true").load("hdfs://rhes564:9000/data/stg/

Re: This works to filter transactions older than certain months

bq. CSV data is stored in an underlying table in Hive (actually created and populated as an ORC table by Spark) How is it possible? On Mon, Mar 28, 2016 at 1:50 AM, Mich Talebzadeh wrote: > Hi, > > A while back I was looking for functional programming to filter out > transactions older > n mont