[
https://issues.apache.org/jira/browse/METRON-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448774#comment-16448774
]
ASF GitHub Bot commented on METRON-1526:
----------------------------------------
Github user merrimanr commented on the issue:
https://github.com/apache/metron/pull/995
To give an idea of how I arrived at the conclusion that other fields are
not a problem, this is how I tested it. I will use currency as an example.
First I added this to our bro schema:
```
<dynamicField name="*.c" type="currency" indexed="true" stored="true"/>
<fieldType name="currency" class="solr.CurrencyField" precisionStep="8"
defaultCurrency="USD" currencyConfig="currency.xml" />
```
I also added the currency.xml file from the example configs that ship with
Solr. Then I added that field to the integration test added in this PR
(SolrUpdateIntegrationTest.suppress_expanded_fields). It ends up looking like:
```
@Test
public void suppress_expanded_fields() throws Exception {
dao = new MultiIndexDao(createDao());
dao.init(accessConfig);
Map<String, Object> fields = new HashMap<>();
fields.put("guid", "bro_1");
fields.put("source.type", SENSOR_NAME);
fields.put("ip_src_port", 8010);
fields.put("long_field", 10000);
fields.put("latitude", 48.5839);
fields.put("score", 10.0);
fields.put("is_alert", true);
fields.put("field.location_point", "48.5839,7.7455");
fields.put("field.c", "1000,EUR");
Document document = new Document(fields, "bro_1", SENSOR_NAME, 0L);
dao.update(document, Optional.of(SENSOR_NAME));
Document indexedDocument = dao.getLatest("bro_1", SENSOR_NAME);
// assert no extra expanded fields are included
Assert.assertEquals(9, indexedDocument.getDocument().size());
}
```
If we get the same number of fields back that we indexed originally, we
know no extra fields are being returned.
> Location field types cause DocValuesField appear more than once error
> ---------------------------------------------------------------------
>
> Key: METRON-1526
> URL: https://issues.apache.org/jira/browse/METRON-1526
> Project: Metron
> Issue Type: Bug
> Reporter: Ryan Merriman
> Assignee: Ryan Merriman
> Priority: Major
>
> While testing [https://github.com/apache/metron/pull/970] I get this error
> when creating a meta alert:
> {code:java}
> Error from server at http://10.0.2.15:8983/solr/bro: Exception writing
> document id bbc150f5-92f8-485d-93cc-11730c1edf31 to the index; possible
> analysis error: DocValuesField
> \"enrichments.geo.ip_dst_addr.location_point_0_coordinate\" appears more than
> once in this document (only one value is allowed per field){code}
> I tracked it down to the fact that multiple fields are returned for a
> location field. For example when a field named
> "enrichments.geo.ip_dst_addr.location_point" is configured in a schema, these
> fields are returned in a query:
> {code:java}
> {
> "enrichments.geo.ip_dst_addr.location_point_0_coordinate": "33.4499",
> "enrichments.geo.ip_dst_addr.location_point_1_coordinate": "-112.0712",
> "enrichments.geo.ip_dst_addr.location_point": "33.4499,-112.0712"
> }
> {code}
> We need a way to either suppress these extra fields when querying or remove
> them before updating a document.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
