ariables?
>
> By the way, what does the spark logs says? Enable debug mode to see what's
> going on in spark-shell when it tries to interact and init HiveContext.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/HiveContext-is-creating-m
java:520)
>>>> at
>>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:398)
>>>> at
>>> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.
>>
>> (ChecksumFileSystem.java:137)
>>>> at
>&
:339)
>> >at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:763)
>> >at
>> >
>> org.apache.hadoop.mapred.LineRecordReader.
>
> (LineRecordReader.java:106)
>> >at
>> >
>> org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
>> >at org.apache.spark.rdd.HadoopRDD$$anon$1.
>
> (HadoopRDD.scala:193)
>> >
>> > At last, I found /user/hive/warehouse/src/kv1.txt was created on the
>> node
>> > where I start spark-shell.
>> >
>> > The spark that I used is pre-built spark1.0.1 for hadoop2.
>> >
>> > Thanks in advance.
>> >
>> >
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/HiveContext-is-creating-metastore-warehouse-locally-instead-of-in-hdfs-tp10838p1.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
d the metastore directories are two different things.
> The
> >> metastore holds the schema information about the tables and will by
> >> default
> >> be a local directory. With javax.jdo.option.ConnectionURL you can
> >> configure it to be something like mysql. The warehouse directory is the
> >> default location where the actual contents of the tables is stored.
> What
> >> directory are seeing created locally?
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/HiveContext-is-creating-metastore-warehouse-locally-instead-of-in-hdfs-tp10838p11024.html
> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
here the actual contents of the tables is stored. What
>> directory are seeing created locally?
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/HiveContext-is-creating-metastore-warehouse-locally-instead-of-in-hdfs-tp10838p11024.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
do.option.ConnectionURL you can
> configure it to be something like mysql. The warehouse directory is the
> default location where the actual contents of the tables is stored. What
> directory are seeing created locally?
--
View this message in context:
http://apache-spark-user-list.1001
--
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/HiveContext-is-creating-metastore-warehouse-locally-instead-of-in-hdfs-tp10838p10871.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
Thanks for the response... hive-site.xml is in the classpath so that doesn't
seem to be the issue.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/HiveContext-is-creating-metastore-warehouse-locally-instead-of-in-hdfs-tp10838p10871.html
Sent from the A
: nikroy16 [mailto:nikro...@gmail.com]
Sent: Tuesday, July 29, 2014 12:51 PM
To: u...@spark.incubator.apache.org
Subject: HiveContext is creating metastore warehouse locally instead of in hdfs
Hi,
Even though hive.metastore.warehouse.dir in hive-site.xml is set to the default
user/hive/warehouse and
-site.xml at the time of initialization. Any help would
be greatly appreciated!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/HiveContext-is-creating-metastore-warehouse-locally-instead-of-in-hdfs-tp10838.html
Sent from the Apache Spark User List mailing list
10 matches
Mail list logo
