[
https://issues.apache.org/jira/browse/METRON-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442742#comment-16442742
]
ASF GitHub Bot commented on METRON-1526:
----------------------------------------
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/995#discussion_r182480647
--- Diff:
metron-platform/metron-solr/src/main/java/org/apache/metron/solr/dao/SolrUpdateDao.java
---
@@ -92,9 +93,17 @@ public void batchUpdate(Map<Document, Optional<String>>
updates) throws IOExcept
}
}
- private SolrInputDocument toSolrInputDocument(Document document) {
+ protected SolrInputDocument toSolrInputDocument(Document document) {
SolrInputDocument solrInputDocument = new SolrInputDocument();
- document.getDocument().forEach(solrInputDocument::addField);
+ document.getDocument().forEach((field, value) -> {
+ if (!isLocationSubField(field)) {
+ solrInputDocument.addField(field, value);
+ }
+ });
return solrInputDocument;
}
+
+ private boolean isLocationSubField(String field) {
+ return field.endsWith(LOCATION_SUB_FIELD_SUFFIX);
--- End diff --
I think there is a better way. Within our schemas there is a "location"
field type that is configured as such:
```
<fieldType name="location" class="solr.LatLonType"
subFieldSuffix="_coordinate"/>
```
The constant in this class matches the "subFieldSuffix" attribute. We
could get column metadata, trace fields back to this type, and remove them
based on the value of the "subFieldSuffix" attribute instead of hardcoding it.
> Location field types cause DocValuesField appear more than once error
> ---------------------------------------------------------------------
>
> Key: METRON-1526
> URL: https://issues.apache.org/jira/browse/METRON-1526
> Project: Metron
> Issue Type: Bug
> Reporter: Ryan Merriman
> Assignee: Ryan Merriman
> Priority: Major
>
> While testing [https://github.com/apache/metron/pull/970] I get this error
> when creating a meta alert:
> {code:java}
> Error from server at http://10.0.2.15:8983/solr/bro: Exception writing
> document id bbc150f5-92f8-485d-93cc-11730c1edf31 to the index; possible
> analysis error: DocValuesField
> \"enrichments.geo.ip_dst_addr.location_point_0_coordinate\" appears more than
> once in this document (only one value is allowed per field){code}
> I tracked it down to the fact that multiple fields are returned for a
> location field. For example when a field named
> "enrichments.geo.ip_dst_addr.location_point" is configured in a schema, these
> fields are returned in a query:
> {code:java}
> {
> "enrichments.geo.ip_dst_addr.location_point_0_coordinate": "33.4499",
> "enrichments.geo.ip_dst_addr.location_point_1_coordinate": "-112.0712",
> "enrichments.geo.ip_dst_addr.location_point": "33.4499,-112.0712"
> }
> {code}
> We need a way to either suppress these extra fields when querying or remove
> them before updating a document.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
