[
https://issues.apache.org/jira/browse/METRON-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444421#comment-16444421
]
ASF GitHub Bot commented on METRON-1526:
----------------------------------------
Github user merrimanr commented on the issue:
https://github.com/apache/metron/pull/995
> Can you confirm that this isn't an issue when you set docValues to false,
like in my previous comment?
I will research this.
> Are date fields polyfields?
I don't know. At this point I am not sure how to determine if a field is a
polyfield or not. The `isPolyField` method is internal and I don't have access
to it in Solrj.
> Is this a problem in 5.x or just 6.x?
I will have to install 5.x and test it out.
> Is there a workaround for dates like there is for latlon in 6.x (e.g.
another type)? If so, wouldn't differentiating schema per version solve the
issue?
I can research this but there may be other fields that cause problems that
we are not aware of yet.
> how do you propose to pull the JSON Patch apart into just the selected
fields?
I'm not following. A patch would only include the field to be added,
changed, or removed.
> Since we will possibly expose these generated fields (perhaps not for
latlon, but possibly for other polytypes), aren't we in the same spot of having
a way to update both a generated field and a derived field if the users can
update derived fields?
I don't understand this question. By moving to partial updates, generated
fields are no longer involved since there is no need to look up a document
before indexing anymore. In the case of metaalert, a patch request would look
like:
```
{
"guid": "some guid",
"index": "bro",
"patch": [
{
"op": "add",
"path": "/metaalerts",
"value": ["some metaalert guid"]
}
],
"sensorType": "bro"
}
```
If for some reason a user is trying to directly update a generated field
that is incorrect and a user error. Would be the same as if they were trying
to update a field that didn't exist in the schema. If a user wants to update a
field that results in generated fields (like latlon), they just update that
field only and let Solr regenerate the internal fields. With our current
approach a generated field is being updated when a valid, unrelated field is
updated. That is the root of the problem.
It's also worth mentioning that partial updates are atomic so we're not
susceptible to race conditions when updating like we are now.
> Location field types cause DocValuesField appear more than once error
> ---------------------------------------------------------------------
>
> Key: METRON-1526
> URL: https://issues.apache.org/jira/browse/METRON-1526
> Project: Metron
> Issue Type: Bug
> Reporter: Ryan Merriman
> Assignee: Ryan Merriman
> Priority: Major
>
> While testing [https://github.com/apache/metron/pull/970] I get this error
> when creating a meta alert:
> {code:java}
> Error from server at http://10.0.2.15:8983/solr/bro: Exception writing
> document id bbc150f5-92f8-485d-93cc-11730c1edf31 to the index; possible
> analysis error: DocValuesField
> \"enrichments.geo.ip_dst_addr.location_point_0_coordinate\" appears more than
> once in this document (only one value is allowed per field){code}
> I tracked it down to the fact that multiple fields are returned for a
> location field. For example when a field named
> "enrichments.geo.ip_dst_addr.location_point" is configured in a schema, these
> fields are returned in a query:
> {code:java}
> {
> "enrichments.geo.ip_dst_addr.location_point_0_coordinate": "33.4499",
> "enrichments.geo.ip_dst_addr.location_point_1_coordinate": "-112.0712",
> "enrichments.geo.ip_dst_addr.location_point": "33.4499,-112.0712"
> }
> {code}
> We need a way to either suppress these extra fields when querying or remove
> them before updating a document.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
