You should not use the scala API to load the file, but the data source API or
the hadoop methods of Spark.
> On 17 Feb 2017, at 10:12, nancy henry <nancyhenry6...@gmail.com> wrote:
>
> Hi All,
>
>
> object Step1 {
> def main(args: Array[String]) = {
>
> val sparkConf = new SparkConf().setAppName("my-app")
> val sc = new SparkContext(sparkConf)
>
> val hiveSqlContext: HiveContext = new
> org.apache.spark.sql.hive.HiveContext(sc)
>
> hiveSqlContext.sql(scala.io.Source.fromFile(args(0)).mkString)
>
> System.out.println("Okay")
>
> }
>
> }
>
>
>
> This is my spark program and my hivescript is at args(0)
>
> $SPARK_HOME/bin/./spark-submit --class com.spark.test.Step1 --master yarn
> --deploy-mode cluster com.spark.test-0.1-SNAPSHOT.jar
> hdfs://spirui-d86-f03-06:9229/samples/testsubquery.hql
>
> but file not found exception is coming
>
> why?
>
> where it is expecting the file to be ?
> in local or hdfs?
> if in hdfs how i should give its path
>
> and is there any better way for hive context than using this to read query
> from a file from hdfs?
>
>
>
>
>
>
>
>
