[
https://issues.apache.org/jira/browse/METRON-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16442920#comment-16442920
]
ASF GitHub Bot commented on METRON-1526:
----------------------------------------
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/995
> This causes a problem in our DAO layer because we don't do partial
updates (we reindex the whole document) and these expanded fields are included
in the updated document.
@merrimanr Can you elaborate on this a bit? I'm not sure I follow the full
scope of the problem. Per @ottobackwards comment, would new fields introduced
by parsers also cause issues? What are the parameters around when a user would
"step in it?" What prophylaxis do we need to ensure, or at least reduce the
risk, that this can happen for other fields?
This makes me think of another topic that's been discussed recently about
message envelopes (e.g. syslog and parser chaining) that potentially wrap
multiple other log types that need to be parsed. If the wrapped messages all
routed to the same index, is that a problem?
> Location field types cause DocValuesField appear more than once error
> ---------------------------------------------------------------------
>
> Key: METRON-1526
> URL: https://issues.apache.org/jira/browse/METRON-1526
> Project: Metron
> Issue Type: Bug
> Reporter: Ryan Merriman
> Assignee: Ryan Merriman
> Priority: Major
>
> While testing [https://github.com/apache/metron/pull/970] I get this error
> when creating a meta alert:
> {code:java}
> Error from server at http://10.0.2.15:8983/solr/bro: Exception writing
> document id bbc150f5-92f8-485d-93cc-11730c1edf31 to the index; possible
> analysis error: DocValuesField
> \"enrichments.geo.ip_dst_addr.location_point_0_coordinate\" appears more than
> once in this document (only one value is allowed per field){code}
> I tracked it down to the fact that multiple fields are returned for a
> location field. For example when a field named
> "enrichments.geo.ip_dst_addr.location_point" is configured in a schema, these
> fields are returned in a query:
> {code:java}
> {
> "enrichments.geo.ip_dst_addr.location_point_0_coordinate": "33.4499",
> "enrichments.geo.ip_dst_addr.location_point_1_coordinate": "-112.0712",
> "enrichments.geo.ip_dst_addr.location_point": "33.4499,-112.0712"
> }
> {code}
> We need a way to either suppress these extra fields when querying or remove
> them before updating a document.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
