Thanks for all the answers! Now, while I agree that simple is better, I think QueryParser should be made thread-safe through the use of a wrapper class (along the line of what Luc suggested) or by using something other than javacc, so this is no longer an issue in future lucene versions.
I caught the error through a load test on a system with low memory and slow processor, which exacerbates the concurrency issues. It was a hard to catch bug: even when it showed up the query results still kinda made sense, because missing or duplicate search terms don't produce complete garbage. I'm a beginner, so I may be missing something big, but if not, many lucene installations out there may be having this problem without knowing it. > -----Mensaje original----- > De: Chris Lamprecht [mailto:[EMAIL PROTECTED] > Enviado el: Miércoles, 24 de Agosto de 2005 07:20 > Para: java-user@lucene.apache.org > Asunto: Re: QueryParser not thread-safe > > > I would just create a new QueryParser for each query. Allocating > short-lived objects is just about free in java, and the time spent > performing the actual search will by far dominate any time spent > constructing QueryParser objects. > > On 8/24/05, Vanlerberghe, Luc <[EMAIL PROTECTED]> wrote: > > Thanks for pointing that out! > > I checked the source and QueryParser is indeed not thread-safe (the > > presence of local variables like jj_lastpos that are used *during* the > > parsing makes this obvious) > > > > Perhaps it should be explicitly mentioned in the javadoc. > > > > The solution I'll probably go for is using ThreadLocal variables > > (available since jdk 1.2, but there was a problem in the sun jvm with > > deallocation when threads stop until jdk 1.4 or so) > > > > an example (not compiled, nor tested): > > > > public class MyQueryFactory { > > public Query createQuery(String expression) { > > return threadLocalQueryParser.get().parse(expression); > > } > > > > private ThreadLocal<QueryParser> threadLocalQueryParser=new > > ThreadLocal<QueryParser>() { > > protected QueryParser initialValue() { > > return new QueryParser("defaultField", new MyAnalyzer()); > > } > > }; > > } > > > > (I used jdk 1.5 generics because the code is clearer, remove the > > <QueryParser> and put a few casts where needed if you want this in 1.4 > > or lower) > > > > initialValue() will be only be called once for every thread that calls > > threadLocalQueryParser.get(). > > > > See the javadoc for java.lang.ThreadLocal > > > > Luc > > > > > > -----Original Message----- > > From: jian chen [mailto:[EMAIL PROTECTED] > > Sent: dinsdag 23 augustus 2005 21:38 > > To: java-user@lucene.apache.org > > Subject: Re: QueryParser not thread-safe > > > > Right. My philosophy is that, make it work, then, make it better. > > > > Don't waste time on something that you are not sure if it would cause > > performance problem. > > > > Jian > > > > On 8/23/05, Paul Elschot <[EMAIL PROTECTED]> wrote: > > > On Tuesday 23 August 2005 19:01, Miles Barr wrote: > > > > On Tue, 2005-08-23 at 13:47 -0300, [EMAIL PROTECTED] wrote: > > > > > Hi! I've been having problems with lucene's QueryParser, apparently > > > > > it is not thread-safe. > > > > > > > > > > That means I can't parse queries in threads where the queryparser > > > > > object > > > > > is created once and reused for each query. If I do, the resulting > > > > > queries may > > > > > have all kinds of weird problems, for example missing terms, > > > > > duplicate terms, etc. > > > > > > > > > > I don't like the idea of creating a new queryparser for each query, > > > > > so I > > > > > am looking for alternatives. Any ideas? > > > > > > > > Using a non-threadsafe object in a threaded environment is fairly > > > > standard in Java, just wrap it in a synchronized block. > > > > > > > > If you don't want all threads waiting on one query parser, create a pool > > > > of them. > > > > > > Or use one parser per thread. > > > > > > Regards, > > > Paul Elschot --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
