Re: NGramTokenizer stops working after about 1000 terms

2010-01-03 Thread Otis Gospodnetic
http://sematext.com/ -- Solr - Lucene - Nutch - Original Message > From: Stefan Trcek > To: java-user@lucene.apache.org > Sent: Mon, December 14, 2009 9:39:34 AM > Subject: NGramTokenizer stops working after about 1000 terms > > Hello > > For a source code (git repo)

NGramTokenizer stops working after about 1000 terms

2009-12-14 Thread Stefan Trcek
Hello For a source code (git repo) search engine I choose to use an ngram analyzer for substring search (something like "git blame"). This worked fine except it didn't find some strings. I tracked it down to the analyzer. When the ngram analyzer yielded about 1000 terms it stopped yielding mor