From the command line can you run:
hadoop fs -ls
And see SomeDir/transactions.csv? It looks like HDFS is not accessible from
wherever you are running spark-itemsimilarity.
Are you trying to run a local version of Spark because the default is "--master
local” This can still access a clustered HDFS if you are configured to access
it from your machine.
On Mar 9, 2015, at 10:35 AM, Jeff Isenhart <[email protected]> wrote:
bump...anybody???
On Wednesday, March 4, 2015 9:22 PM, Jeff Isenhart
<[email protected]> wrote:
I am having issue getting a simple itemsimilarity example to work. I know
hadoop is up and functional (ran the example mapreduce program anyway)
But when I run either of these
./mahout spark-itemsimilarity -i "SomeDir/transactions.csv" -o
"hdfs://localhost:9000/users/someuser/output" -fc 1 -ic 2
./mahout spark-itemsimilarity -i "SomeDir/transactions.csv" -o "SomeDir/output"
-fc 1 -ic 2
and get
Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2421) at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428) at
org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88) at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467) at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367) at
org.apache.hadoop.fs.FileSystem.get(FileSystem.java:166) at
org.apache.mahout.common.HDFSPathSearch.<init>(HDFSPathSearch.scala:36) at
org.apache.mahout.drivers.ItemSimilarityDriver$.readIndexedDatasets(ItemSimilarityDriver.scala:152)
at
org.apache.mahout.drivers.ItemSimilarityDriver$.process(ItemSimilarityDriver.scala:213)
at
org.apache.mahout.drivers.ItemSimilarityDriver$$anonfun$main$1.apply(ItemSimilarityDriver.scala:116)
at
org.apache.mahout.drivers.ItemSimilarityDriver$$anonfun$main$1.apply(ItemSimilarityDriver.scala:114)
at scala.Option.map(Option.scala:145) at
org.apache.mahout.drivers.ItemSimilarityDriver$.main(ItemSimilarityDriver.scala:114)
at
org.apache.mahout.drivers.ItemSimilarityDriver.main(ItemSimilarityDriver.scala)
I am guessing there are some config settings I am missing
Usingmahout 1.0 Snapshothadoop 2.6.0
