[
https://issues.apache.org/jira/browse/NUTCH-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sebastian Nagel resolved NUTCH-2533.
------------------------------------
Resolution: Implemented
Hi [~krzysztofmadejski], now Nutch (2.x and 1.x) will log what is going on, but
as expected injecting from / will cause just troubles:
{noformat}
% bin/nutch inject /
InjectorJob: Injecting urlDir: /
Skipped non-file input in /: file:/dev
Skipped non-file input in /: file:/lost+found
Skipped non-file input in /: file:/usr
Skipped non-file input in /: file:/sbin
Skipped non-file input in /: file:/snap
Skipped non-file input in /: file:/mnt
Injecting seed URL file file:/vmlinuz
...
Skipping ^@^A^@^@^@...: java.net.MalformedURLException: ...
...
{noformat}
> Injector: NullPointerException if seed URL dir contains non-file entries
> ------------------------------------------------------------------------
>
> Key: NUTCH-2533
> URL: https://issues.apache.org/jira/browse/NUTCH-2533
> Project: Nutch
> Issue Type: Bug
> Components: injector
> Affects Versions: 2.3.1, 1.14
> Reporter: Krzysztof Madejski
> Assignee: Sebastian Nagel
> Priority: Minor
> Fix For: 2.4, 1.15
>
>
> I'm following https://wiki.apache.org/nutch/Nutch2Tutorial
>
> I've run `./nutch inject /` and I've got the following error:
> {noformat}
> InjectorJob: starting at 2018-03-12 11:59:05
> InjectorJob: Injecting urlDir: /
> InjectorJob: Using class org.apache.gora.hbase.store.HBaseStore as the Gora
> storage class.
> InjectorJob: java.lang.NullPointerException
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getBlockIndex(FileInputFormat.java:442)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:411)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
> at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:115)
> at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:231)
> at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:252)
> at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:275)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
