[
https://issues.apache.org/jira/browse/NUTCH-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433789#comment-16433789
]
ASF GitHub Bot commented on NUTCH-2552:
---------------------------------------
sebastian-nagel opened a new pull request #315: NUTCH-2552 CrawlDbReader -topN
fails
URL: https://github.com/apache/nutch/pull/315
- update Hadoop properties (deprecated / old MapReduce API)
- includes fix for
NUTCH-1228 Change mapred.task.timeout to mapreduce.task.timeout in fetcher
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> CrawlDbReader -topN fails
> -------------------------
>
> Key: NUTCH-2552
> URL: https://issues.apache.org/jira/browse/NUTCH-2552
> Project: Nutch
> Issue Type: Bug
> Components: crawldb
> Affects Versions: 1.15
> Reporter: Sebastian Nagel
> Priority: Critical
> Fix For: 1.15
>
>
> {noformat}
> % bin/nutch readdb crawldb -topN 50 crawldb_topn
> CrawlDb topN: starting (topN=50, min=0.0)
> CrawlDb db: crawl/crawldb
> CrawlDb topN: collecting topN scores.
> CrawlDbReader job did not succeed, job status:FAILED, reason: NA
> Exception in thread "main" java.lang.RuntimeException: CrawlDbReader job did
> not succeed, job status:FAILED, reason: NA
> at
> org.apache.nutch.crawl.CrawlDbReader.processTopNJob(CrawlDbReader.java:853)
> {noformat}
> The hadoop.log shows the reason
> {noformat}
> 2018-04-09 10:04:16,435 WARN mapred.LocalJobRunner - job_local1653923841_0002
> java.lang.Exception: java.lang.NumberFormatException: null
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
> Caused by: java.lang.NumberFormatException: null
> at java.lang.Integer.parseInt(Integer.java:542)
> at java.lang.Integer.parseInt(Integer.java:615)
> at
> org.apache.nutch.crawl.CrawlDbReader$CrawlDbTopNReducer.setup(CrawlDbReader.java:370)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:168)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
> {noformat}
> Caused by NUTCH-2375: the property {{mapred.job.reduces}} must be updated by
> {{mapreduce.job.reduces}}.
> Note: Should check all occurrences of this property and similars ones
> ({{mapred.job.*}}).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
