Thanks for reply. Here are the facts:
1. I am using mahout shell command and not a java program. So I am not
passing any arguments to map function.
2. I am using hadoop. Input training file is loaded in hadoop. It is a tab
separated 'u1.base' file of MovieLens dataset.
It is something like below. All users are there along with whatever
ratings they have given.
1 1 5
1 2 3
1 3 4
1 4 3
1 5 3
:
:
2 1 4
2 10 2
2 14 4
:
:
3. I use the following mahout command to build model:
mahout parallelALS --input /user/ashokharnal/u1.base --output
/user/ashokharnal/u1.out --lambda 0.1 --implicitFeedback true --alpha
0.8 --numFeatures 15 --numIterations 10 --numThreadsPerSolver 1
--tempDir /tmp/ratings
4. My test file is just two-lines tab-separated file as below:
1 1
2 1
5. This file is converted to sequence file using the following mahout command:
mahout seqdirectory -i /user/ashokharnal/ufind2.test -o
/user/ashokharnal/seqfiles
6. I then run the following mahout command:
mahout recommendfactorized --input /user/ashokharnal/seqfiles
--userFeatures /user/ashokharnal/u1.out/U/ --itemFeatures
/user/akh/u1.out/M/ --numRecommendations 1 --output /tmp/reommendation
--maxRating 1
7. I am using CentOS 6.5 with Cloudera 5.2 installed.
The error messages are as below:
14/11/24 18:06:48 INFO mapred.MapTask: Processing split:
hdfs://master:8020/user/ashokharnal/seqfiles/part-m-00000:0+195
14/11/24 18:06:49 INFO zlib.ZlibFactory: Successfully loaded &
initialized native-zlib library
14/11/24 18:06:49 INFO compress.CodecPool: Got brand-new decompressor [.deflate]
14/11/24 18:06:49 INFO mapred.LocalJobRunner: Map task executor complete.
14/11/24 18:06:49 WARN mapred.LocalJobRunner: job_local1177125820_0001
java.lang.Exception: java.lang.RuntimeException:
java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast
to org.apache.hadoop.io.IntWritable
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:406)
Caused by: java.lang.RuntimeException: java.lang.ClassCastException:
org.apache.hadoop.io.Text cannot be cast to
org.apache.hadoop.io.IntWritable
at
org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper.run(MultithreadedMapper.java:151)
at
org.apache.mahout.cf.taste.hadoop.als.MultithreadedSharingMapper.run(MultithreadedSharingMapper.java:60)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text
cannot be cast to org.apache.hadoop.io.IntWritable
at
org.apache.mahout.cf.taste.hadoop.als.PredictionMapper.map(PredictionMapper.java:44)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
at
org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper$MapRunner.run(MultithreadedMapper.java:268)
14/11/24 18:06:49 INFO mapred.JobClient: map 0% reduce 0%
14/11/24 18:06:49 INFO mapred.JobClient: Job complete: job_local1177125820_0001
14/11/24 18:06:49 INFO mapred.JobClient: Counters: 0
14/11/24 18:06:49 INFO driver.MahoutDriver: Program took 2529 ms
(Minutes: 0.04215)
14/11/24 18:06:49 ERROR hdfs.DFSClient: Failed to close inode 24733
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
No lease on
/tmp/reommendation/_temporary/_attempt_local1177125820_0001_m_000000_0/part-m-00000
(inode 24733): File does not exist. Holder
DFSClient_NONMAPREDUCE_157704469_1 does not have any open files.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3319)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:3407)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:3377)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.complete(NameNodeRpcServer.java:673)
at
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.complete(AuthorizationProviderProxyClientProtocol.java:219)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.complete(ClientNamenodeProtocolServerSideTranslatorPB.java:520)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:587)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1026)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
at org.apache.hadoop.ipc.Client.call(Client.java:1411)
at org.apache.hadoop.ipc.Client.call(Client.java:1364)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy16.complete(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.complete(ClientNamenodeProtocolTranslatorPB.java:435)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy17.complete(Unknown Source)
at
org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:2180)
at
org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2164)
at
org.apache.hadoop.hdfs.DFSClient.closeAllFilesBeingWritten(DFSClient.java:908)
at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:926)
at
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:861)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2687)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2704)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)
Sorry for bothering
Ashok Kumar Harnal
On 24 November 2014 at 15:50, Divyang Shah <[email protected]>
wrote:
> hello, problem is in map method argument you have passed is not
> matching with specified in job configuration. so, match both of them.
>
>
> On Sunday, 23 November 2014 8:31 AM, Ashok Harnal <
> [email protected]> wrote:
>
>
> I use mahout 0.7 installed in Cloudera. After creating user-feature and
> item-feature matrix in hdfs, I run the following command:
>
> mahout recommendfactorized --input /user/ashokharnal/seqfiles
> --userFeatures $res_out_file/U/ --itemFeatures $res_out_file/M/
> --numRecommendations 1 --output $reommendation --maxRating 1
>
> After some time, I get the following error:
>
> :
> :
> 14/11/23 08:28:20 INFO mapred.LocalJobRunner: Map task executor complete.
> 14/11/23 08:28:20 WARN mapred.LocalJobRunner: job_local954305987_0001
> java.lang.Exception: java.lang.RuntimeException:
> java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to
> org.apache.hadoop.io.IntWritable
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:406)
> Caused by: java.lang.RuntimeException: java.lang.ClassCastException:
> org.apache.hadoop.io.Text cannot be cast to
> org.apache.hadoop.io.IntWritable
> at
>
> org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper.run(MultithreadedMapper.java:151)
> at
>
> org.apache.mahout.cf.taste.hadoop.als.MultithreadedSharingMapper.run(MultithreadedSharingMapper.java:60)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
> at
>
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.lang.ClassCastException: org.apache.hadoop.io.Text cannot
> be cast to org.apache.hadoop.io.IntWritable
> at
>
> org.apache.mahout.cf.taste.hadoop.als.PredictionMapper.map(PredictionMapper.java:44)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
> at
>
> org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper$MapRunner.run(MultithreadedMapper.java:268)
>
>
> Not sure what is wrong.
> Request help.
>
> Ashok Kumar Harnal
>
>
>
>
> --
> Visit my blog at: http://ashokharnal.wordpress.com/
>
>
>
>
--
Visit my blog at: http://ashokharnal.wordpress.com/
