Re: a query for a special AND?

2007-09-19 Thread Mohammad Norouzi
Sorry Paul I just hurried in replying ;) I read the documents of Lucene about query syntax and I figured out the what is the difference but my problem is different, this is preoccupied my mind and I am under pressure to solve this problem, after analyzing the results I get, now I think we need a "g

Re: a query for a special AND?

2007-09-19 Thread Mohammad Norouzi
Hi Paul, would you tell me what is the difference between AND and + ? I tried both but get different result with AND I get 1777 documents and with + I get nearly 25000 ? On 9/17/07, Paul Elschot <[EMAIL PROTECTED]> wrote: > > On Monday 17 September 2007 11:40, Mohammad Norouzi wrote: > > Hi > > I

Re: thread safe shared IndexSearcher

2007-09-19 Thread Mark Miller
The method is synched, but this is because each thread *does* share the same Searcher. To maintain a cache of searchers across multiple threads, you've got to sync -- to reference count, you've got to sync. The performance hit of LuceneIndexAcessor is pretty minimal for its functionality, and f

Re: thread safe shared IndexSearcher

2007-09-19 Thread Jay Yu
Mark, After reading the implementation of LuceneIndexAccessor.getSearcher(), I realized that the method is synchronized and wait for writingDirector to be released. That means if we getSearcher for each query in each thread, there might be a contention and performance hit. In fact, even the me

Re: thread safe shared IndexSearcher

2007-09-19 Thread Jay Yu
Thanks for your detailed explanation of the issues and your solutions. It seems that LuceneIndexAccessor is worth trying first before I implement other locking mechanism to ensure proper order. I will appreciate it very much if you'd like your extension with us. Jay Mark Miller wrote: Ill res

Re: thread safe shared IndexSearcher

2007-09-19 Thread Mark Miller
Ill respond a point at a time: 1. ** Hi Maik, So what happens in this case: IndexAccessProvider accessProvider = new IndexAccessProvider(directory, analyzer); LuceneIndexAccessor accessor = new LuceneIndexAccessor(accessProvider); accessor.open(); IndexWriter wri

Re: Lucene implmentation question

2007-09-19 Thread Zach Bailey
I am a little suspicious of your classpath, still. You are using ',' to delimit classpath entries, and from my experience generally this character should either be ';' (windows) or ':' (*nix). You can easily discover this character by rolling a quick java program containing the following line:

Re: thread safe shared IndexSearcher

2007-09-19 Thread Jay Yu
Mark, thanks for sharing your insight and experience about LuceneIndexAccessor! I remember seeing some people reporting some issues about it, such as: http://www.archivum.info/[EMAIL PROTECTED]/2005-05/msg00114.html http://issues.apache.org/bugzilla/show_bug.cgi?id=34995#c3 Have those issues bee

Re: thread safe shared IndexSearcher

2007-09-19 Thread Mark Miller
I use option 3 extensivley and find it very effective. There is a tweak or two required to get it to work right with MultiSearchers, but other than that, the code is great. I have built a lot on top of it. I'm on the list all the time and would be happy to answer any questions you have in regards t

Re: Lucene implmentation question

2007-09-19 Thread warnockm
Thanks for your reply. I tried your suggestion of including the .jar files individually in the classpath, but i didn't have any luck: [EMAIL PROTECTED] ~]# java -cp /usr/java/default/lib/lucene-core-2.2.0.jar,/usr/java/default/lib/lucene-demos-2.2.0.jar org.apache.lucene.demo.IndexFiles /opt/luc

thread safe shared IndexSearcher

2007-09-19 Thread Jay Yu
In a multithread app like web app, a shared IndexSearcher could throw a AlreadyClosedException when another thread is trying to update the underlying IndexReader by closing the shared searcher after the index is updated. Searching over the past discussions on this mailing list, I found several

Re: field name limitations?

2007-09-19 Thread Yonik Seeley
Should be fine. The biggest issue may be the number of fields you are likely to have. They are all kept in an in-memory map. Also, during segment merging, the field properties are merged in a linear fashion by stepping over each one. -Yonik On 9/19/07, Karl Wettin <[EMAIL PROTECTED]> wrote: > I'

Re: Lucene implmentation question

2007-09-19 Thread Raymond Kroeker
Hi Matt, You'll want to point the classpath to the lib files and not their containing folder ie: java_dir/lib/core.jar:java_dir/lib/demo.jar Raymond On 9/19/07, warnockm <[EMAIL PROTECTED]> wrote: > > > This sounds like a simple problem but i cannot for the life of me figure > it > out. Am ru

Re: Lucene implmentation question

2007-09-19 Thread Zach Bailey
Also note it appears IndexFiles should be in a Lucene "demo" JAR, so make sure you have that and specify it on your classpath in addition to the regular Lucene JAR. Sorry for the confusion. -Zach Zach Bailey wrote: My next recommendation would be to manually specify the classpath by using th

Re: Lucene implmentation question

2007-09-19 Thread Zach Bailey
My next recommendation would be to manually specify the classpath by using the -cp or -classpath arguments, for example: /usr/java/default/bin/java -cp /usr/java/default/lib/lucene.jar org.apache.lucene.demo.IndexFiles /opt/lucene/src/ By the way, you are aware that IndexFiles is not included

Re: Lucene implmentation question

2007-09-19 Thread warnockm
Thanks. I did not realize that. I fixed it by putting in the path, but am getting the same error: [EMAIL PROTECTED] /]# /usr/java/default/bin/java org.apache.lucene.demo.IndexFiles /opt/lucene/src/ Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/lucene/demo/IndexFiles Za

Re: Lucene implmentation question

2007-09-19 Thread Zach Bailey
Are you aware you're using GCJ and not the Sun JVM/JDK? Your initial paragraph seems to imply you think you are using a Sun product, but your stack trace shows otherwise. Perhaps that has something to do with it - you're not actually running the "java" you think you are? -Zach warnockm wrote:

Lucene implmentation question

2007-09-19 Thread warnockm
This sounds like a simple problem but i cannot for the life of me figure it out. Am running Fedora 7, JDK 1.6, and Tomcat 6.0 (although not being used for this). I download Lucene and followed the instructions that said to place the core and demo .jar files in my classpath. My classpath is set

field name limitations?

2007-09-19 Thread Karl Wettin
I'm considering using complete class xpaths as field names. These might end up beeing several hundred letters long and contain some wierd letters. Should I expect running in to problems? Field names might look something like this: the.absolute:Class.Path/@attribute/tokenizationStrategyA the.

Re: Java Heap Space -Out Of Memory Error

2007-09-19 Thread Sebastin
HI testn, I used to search now 3 folders of total size 1.5.GB.still it consumes lot of memory for every search.i used to close all the IndexReader once i finish the search.i optimize the FIles using Luke.when i set the IndexSearcher object as application level object,its not possibl