context:
http://apache-spark-user-list.1001560.n3.nabble.com/Hadoop-Input-Format-newAPIHadoopFile-tp2860p10762.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Thanks . it worked..
Very basic question, i have created custominput format e.g. stock. How do
I refer this class as custom inputformat. I.e. where to keep this class on
linux folder. Do i need to add this jar if so how .
I am running code through spark-shell.
Thanks
Pari
On 19-Mar-2014 7:35 pm,
The correct import statement is "import
org.apache.hadoop.mapreduce.lib.input.TextInputFormat".
Best Regards,
Shixiong Zhu
2014-03-19 18:46 GMT+08:00 Pariksheet Barapatre :
> Seems like import issue, ran with HadoopFile and it worked. Not getting
> import statement for textInputFormat class loc
Seems like import issue, ran with HadoopFile and it worked. Not getting
import statement for textInputFormat class location for new API.
Can anybody help?
Thanks
Pariksheet
On 19 March 2014 16:05, Bertrand Dechoux wrote:
> I don't know the Spark issue but the Hadoop context is clear.
>
> old
I don't know the Spark issue but the Hadoop context is clear.
old api -> org.apache.hadoop.mapred
new api -> org.apache.hadoop.mapreduce
You might only need to change your import.
Regards
Bertrand
On Wed, Mar 19, 2014 at 11:29 AM, Pariksheet Barapatre wrote:
> Hi,
>
> Trying to read HDFS fi
Hi,
Trying to read HDFS file with TextInputFormat.
scala> import org.apache.hadoop.mapred.TextInputFormat
scala> import org.apache.hadoop.io.{LongWritable, Text}
scala> val file2 =
sc.newAPIHadoopFile[LongWritable,Text,TextInputFormat]("hdfs://
192.168.100.130:8020/user/hue/pig/examples/data/sonn
