Re: This works to filter transactions older than certain months

Forgot to mention Spark 1.6.1 Hive 2.0 Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com On 28 March 2016

Re: This works to filter transactions older than certain months

Snippet. import org.apache.spark.sql.functions._ import java.sql.{Date, Timestamp} val HiveContext = new org.apache.spark.sql.hive.HiveContext(sc) val df = sqlContext.read.format("com.databricks.spark.csv").option("inferSchema", "true").option("header", "true").load("hdfs://rhes564:9000/data/stg/

Re: This works to filter transactions older than certain months

bq. CSV data is stored in an underlying table in Hive (actually created and populated as an ORC table by Spark) How is it possible? On Mon, Mar 28, 2016 at 1:50 AM, Mich Talebzadeh wrote: > Hi, > > A while back I was looking for functional programming to filter out > transactions older > n mont

This works to filter transactions older than certain months

Hi, A while back I was looking for functional programming to filter out transactions older > n months etc. This turned out to be pretty easy. I get today's day as follows var today = sqlContext.sql("SELECT FROM_unixtime(unix_timestamp(), '-MM-dd') ").collect.apply(0).getString(0) CSV data