jnioche commented on code in PR #1242:
URL:
https://github.com/apache/incubator-stormcrawler/pull/1242#discussion_r1642425446
##########
external/solr/cores/status/conf/schema.xml:
##########
@@ -25,5 +25,7 @@ under the License.
<field name="status" type="string" indexed="true" stored="true"
required="true"/>
<field name="nextFetchDate" type="pdate" stored="true" indexed="true"/>
<dynamicField name="metadata*" type="string" indexed="true"
stored="true" multiValued="true"/>
+ <field name="key" type="string" stored="true" indexed="true"/>
Review Comment:
@mvolikas why not set the key field to multiValued = false?
I assume the dynamic field is to allow users to set a different name than
'key'
##########
external/solr/src/main/java/org/apache/stormcrawler/solr/persistence/StatusUpdaterBolt.java:
##########
@@ -83,6 +126,23 @@ public void store(
doc.setField(String.format(Locale.ROOT, "%s.%s", mdPrefix, key),
values);
}
+ String partitionKey = partitioner.getPartition(url, metadata);
+ if (partitionKey == null) {
+ partitionKey = "_DEFAULT_";
+ }
+
+ // store routing key in metadata?
+ if (StringUtils.isNotBlank(fieldNameForRoutingKey) &&
routingFieldNameInMetadata) {
Review Comment:
given that this will not change from one document to the other, why not
determine the field name to use once and for all instead of doing it for every
document?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
