[
https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13859006#comment-13859006
]
Lance Norskog commented on LUCENE-2899:
---------------------------------------
All fair criticisms.
About UIMA: clearly it is much more advanced than this design, but I'm not
smart enough to use it :) I've tried to put together something useful (a few
times) and each time was completely confused. I learn by example, and the
examples are limited. Also there is very little traffic on the mailing lists
etc. about UIMA.
About payloads v.s. internal attributes: the examples don't use this feature,
but payloads are stored in the index. This supports a question-answering
system. Add PERSON payloads with all records, then search for "word X AND
'payload PERSON anywhere'" when someone says "who is X". This does the tagging
during indexing, but not searching. A better design would be to add PERSON as a
synonym rather than a payload. I also don't see much traffic about payloads.
About doing this in the analysis pipeline v.s. upstream: yes, upstream request
processors are the right place for this. In Solr. URPs don't exist in ES or
just plain Lucene coding.
> Add OpenNLP Analysis capabilities as a module
> ---------------------------------------------
>
> Key: LUCENE-2899
> URL: https://issues.apache.org/jira/browse/LUCENE-2899
> Project: Lucene - Core
> Issue Type: New Feature
> Components: modules/analysis
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 4.7
>
> Attachments: LUCENE-2899-RJN.patch, LUCENE-2899.patch,
> OpenNLPFilter.java, OpenNLPTokenizer.java
>
>
> Now that OpenNLP is an ASF project and has a nice license, it would be nice
> to have a submodule (under analysis) that exposed capabilities for it. Drew
> Farris, Tom Morton and I have code that does:
> * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it
> would have to change slightly to buffer tokens)
> * NamedEntity recognition as a TokenFilter
> We are also planning a Tokenizer/TokenFilter that can put parts of speech as
> either payloads (PartOfSpeechAttribute?) on a token or at the same position.
> I'd propose it go under:
> modules/analysis/opennlp
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
