[
https://issues.apache.org/jira/browse/SOLR-11741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16315469#comment-16315469
]
Abhishek Kumar Singh edited comment on SOLR-11741 at 1/7/18 8:56 PM:
---------------------------------------------------------------------
Uploading patch [^SOLR-11741-temp.patch] with above implementation within
*AddSchemaFieldsUpdateProcessorFactory* itself.
Have added a config param called mode.
The above URP will just *Train The Schema* when *mode=train* . By default
*mode=update*, i.e. update the schema as usual.
This patch is temporary because it still needs test cases. Also, currently the
state is being stored in-memory, in a map.
Have to move that to the zookeeper. Will update that design in my next comments.
was (Author: abhidemon):
Uploading patch with above implementation within
*AddSchemaFieldsUpdateProcessorFactory* itself.
Have added a config param called mode.
The above URP will just *Train The Schema* when *mode=train* . By default
*mode=update*, i.e. update the schema as usual.
This patch is temporary because it still needs test cases. Also, currently the
state is being stored in-memory, in a map.
Have to move that to the zookeeper. Will update that design in my next comments.
> Offline training mode for schema guessing
> -----------------------------------------
>
> Key: SOLR-11741
> URL: https://issues.apache.org/jira/browse/SOLR-11741
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Ishan Chattopadhyaya
> Attachments: SOLR-11741-temp.patch, screenshot-1.png, screenshot-3.png
>
>
> Our data driven schema guessing doesn't work under many situations. For
> example, if the first document has a field with value "0", it is guessed as
> Long and subsequent fields with "0.0" are rejected. Similarly, if the same
> field had alphanumeric contents for a latter document, those documents are
> rejected. Also, single vs. multi valued field guessing is not ideal.
> Proposing an offline training mode where Solr accepts bunch of documents and
> returns a guessed schema (without indexing). This schema can then be used for
> actual indexing. I think the original idea is from Hoss.
> I think initial implementation can be based on an UpdateRequestProcessor. We
> can hash out the API soon, as we go along.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
