Tommy, I'm sure that I don't fully understand your use case and your data. Some thoughts:
1) I assume that fuzzy term search (edit distance <= 2) isn't meeting your needs or else you wouldn't have gone the ngram route. If fuzzy term search + phrase/proximity search would meet your needs, see if ComplexPhraseQueryParser would work (although it looks like you're already building your own queries). 2) Would it make sense to modify NGramFilter so that it outputs a bigram for a two letter term and a unigram for a one letter term? Might be messy...and "ab" in this scenario would never match "abc" 3) Would it make sense to pad your terms behind the scenes with "##"...this would add bloat, but not nearly as much as variable gram sizes with 1<= n <=3 ab -> ##ab## yields trigrams ##a, #ab, ab#, b## 4) How partial and what types of partial do you need? This is related to 1). If minimum edit distance is sufficient; use it, especially with the blazing fast automaton (thank you, Robert Muir). If you have a smallish dataset you might consider allowing leading wildcards so that you could easily find all words, for example, containing abc with *abc*. If your dataset is larger, you might consider something like ReversedWildcardFilterFactory (Solr) to speed this type of matching. I look forward to other opinions from the list. -----Original Message----- From: Becker, Thomas [mailto:thomas.bec...@netapp.com] Sent: Thursday, July 18, 2013 3:55 PM To: java-user@lucene.apache.org Subject: Partial word match using n-grams One of our main use-cases for search is to find objects based on partial name matches. I've implemented this using n-grams and it works pretty well. However we're currently using trigrams and that causes an interesting problem when searching for things like "abc ab" since we first split on whitespace and then construct PhraseQuerys containing each trigram yielded by the "word". Obviously we cannot get a trigram out of "ab". So our choices would seem to be either discard this part of the search term which seems unwise, or to reduce the minimum n-gram size. But I'm slightly concerned about the resulting bloat in both the of number of Terms stored in the index as well as contained in queries. Is this something I should be concerned about? It just "feels" like a query for the word "abcdef" shouldn't require a PhraseQuery of 15 terms (assuming n-grams 1,3). Is this the best way to do partial word matches? Thanks in advance. -Tommy --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
