jnioche commented on code in PR #2112:
URL: https://github.com/apache/stormcrawler/pull/2112#discussion_r3912873343
##########
core/src/main/java/org/apache/stormcrawler/parse/ParseResult.java:
##########
@@ -66,19 +66,51 @@ public void setOutlinks(List<Outlink> outlinks) {
}
/**
- * @return An existent instance of Parse for the given URL or an empty one
if none can be found,
- * useful to avoid unnecessary checks in the parse plugins
+ * Convenience accessor which returns the ParseData for the given URL,
creating an empty entry
+ * in this ParseResult if none can be found, useful to avoid unnecessary
checks in the parse
+ * plugins. Note that looking up a URL which has not been parsed modifies
this ParseResult: the
+ * entry created for it is treated like any other document and gets
emitted by the parser bolts.
+ *
+ * @deprecated use {@link #getIfPresent(String)} for a read-only lookup or
{@link
+ * #getOrCreate(String)} to create an entry, so that the intent is
explicit at the call site
+ * @return An existent instance of Parse for the given URL or an empty one
if none can be found
*/
+ @Deprecated
Review Comment:
the next release will be a major one, we might as well get rid of the method
straight away.
##########
core/src/main/java/org/apache/stormcrawler/bolt/JSoupParserBolt.java:
##########
@@ -493,10 +493,20 @@ public void execute(Tuple tuple) {
}
// emit each document/subdocument in the ParseResult object
- // there should be at least one ParseData item for the "parent" URL
-
+ // there should be at least one ParseData item for the "parent" URL.
Review Comment:
if we totally deprecate get() - we will probably never come across those
empty docs. Happy to keep the check and the log message but probably could get
rid of that lengthy AI generated Javadoc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
