> On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/conf/nutch-default.xml, line 29 > > <https://reviews.apache.org/r/31579/diff/1/?file=881552#file881552line29> > > > > Is this needed?
No, it is not. I used it for testing purporses. I restored it to the original value. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/conf/nutch-default.xml, line 189 > > <https://reviews.apache.org/r/31579/diff/1/?file=881552#file881552line189> > > > > Is this needed? No, it is not. I used it for testing purporses. I restored it to the original value. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/AbstractCommonCrawlFormat.java, line 1 > > <https://reviews.apache.org/r/31579/diff/1/?file=881554#file881554line1> > > > > please add Apache header. Apache License 2 added. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 209 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line209> > > > > reminder to remove later. Comment removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 269 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line269> > > > > reminder, remove later. Comment removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 282 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line282> > > > > reminder, remove later. Comment removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 307 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line307> > > > > eminder, remove later. Comment removed. DataOutputStream is never used in FileDumper.java. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 339 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line339> > > > > reminder, remove later. Comment removed. ByteArrayInputStream is never used id FileDumper.java. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 347 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line347> > > > > remove later. Comment removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 350 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line350> > > > > reminder, remove later. Comment removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 355 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line355> > > > > reminder remove later. Comment removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 409 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line409> > > > > remove Commented code has been removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 463 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line463> > > > > remove later. Comment removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java, line 477 > > <https://reviews.apache.org/r/31579/diff/1/?file=881555#file881555line477> > > > > remove later. Comment removed. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormat.java, line 1 > > <https://reviews.apache.org/r/31579/diff/1/?file=881556#file881556line1> > > > > Apache license headers. Apache license 2 added. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatFactory.java, line 1 > > <https://reviews.apache.org/r/31579/diff/1/?file=881557#file881557line1> > > > > Apache license headers. Apache license 2 added. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJackson.java, line 1 > > <https://reviews.apache.org/r/31579/diff/1/?file=881558#file881558line1> > > > > apache 2 headers Apache license 2 added. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJackson.java, line > > 122 > > <https://reviews.apache.org/r/31579/diff/1/?file=881558#file881558line122> > > > > maybe have a LOG.warning here? LOG.warn added. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJackson.java, line > > 232 > > <https://reviews.apache.org/r/31579/diff/1/?file=881558#file881558line232> > > > > maybe LOG.warning() LOG.warn added. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJackson.java, line > > 235 > > <https://reviews.apache.org/r/31579/diff/1/?file=881558#file881558line235> > > > > throw an Exception. Ok. Now, it throws an Exception in case of failure. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJettinson.java, line > > 1 > > <https://reviews.apache.org/r/31579/diff/1/?file=881559#file881559line1> > > > > apache 2 headers Apache License 2 addded. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJettinson.java, line > > 143 > > <https://reviews.apache.org/r/31579/diff/1/?file=881559#file881559line143> > > > > LOG.warning LOG.warn added. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJettinson.java, line > > 80 > > <https://reviews.apache.org/r/31579/diff/1/?file=881559#file881559line80> > > > > throw exception Ok. Now, it throws an Exception in case of failure. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJettinson.java, line > > 77 > > <https://reviews.apache.org/r/31579/diff/1/?file=881559#file881559line77> > > > > LOG.warning LOG.warn added. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJettinson.java, line > > 146 > > <https://reviews.apache.org/r/31579/diff/1/?file=881559#file881559line146> > > > > throw Exception Ok. Now, it throws an Exception in case of failure. > On Mar. 2, 2015, 2:45 a.m., Chris Mattmann wrote: > > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatSimple.java, line 1 > > <https://reviews.apache.org/r/31579/diff/1/?file=881560#file881560line1> > > > > apache 2 headers Apache License 2 addded. - Giuseppe ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31579/#review74717 ----------------------------------------------------------- On Mar. 2, 2015, 2:45 a.m., Giuseppe Totaro wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/31579/ > ----------------------------------------------------------- > > (Updated Mar. 2, 2015, 2:45 a.m.) > > > Review request for nutch, Lewis McGibbney and Chris Mattmann. > > > Bugs: NUTCH-1949 > https://issues.apache.org/jira/browse/NUTCH-1949 > > > Repository: nutch > > > Description > ------- > > Patch fo NUTCH-1949: first version of the CommonCrawlDataDumper tool that > maps Nutch data into Common Crawl format. > > > Diffs > ----- > > trunk/conf/nutch-default.xml 1662875 > trunk/src/bin/nutch 1662875 > trunk/src/java/org/apache/nutch/tools/AbstractCommonCrawlFormat.java > PRE-CREATION > trunk/src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java > PRE-CREATION > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormat.java PRE-CREATION > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatFactory.java > PRE-CREATION > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJackson.java > PRE-CREATION > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatJettinson.java > PRE-CREATION > trunk/src/java/org/apache/nutch/tools/CommonCrawlFormatSimple.java > PRE-CREATION > trunk/src/java/org/apache/nutch/tools/FileDumper.java 1662875 > > Diff: https://reviews.apache.org/r/31579/diff/ > > > Testing > ------- > > Tested locally against Nutch segments. > > > Thanks, > > Giuseppe Totaro > >
