Agreed. Perhaps I will abandon the static init. I really only put it as
an option due to your synchronized cost concerns (a preload allows non
synched read only access to the indexaccessor cache). Due keep in mind
that you don't have to use it though...if you dont preload, accessors
are created
I agree with you on the compromise aspect of the design.
In particular, I think it's hard to preload all the index accessors in
the static init while allowing users specify the analyzer for each dir
without requiring complicated config file ans using reflection.
So a good compromise might be aba
I think its just a compromise in the design, though it could be
improved. You only ever want a single Writer at a time on the index.
Those two flags are really just hints for when a Writer is first
opened...should it auto-commit and should it overwrite/create...if a
thread tries to writer concu
Mark,
Looking at your implementation of the DefaultIndexAccessor regarding the
writer, I think there could be a problem: you have only one cached
writer but the getWriter(boolean, boolean) allows 2 booleans, so
ideally, you need 4 cached writer. Otherwise if one starts with a writer
that over
Ah, thanks for catching that. One of the pieces I did not finish...the
keyword analyzer was placeholder code.
I will take your comments into account and update the code.
I have some other pieces to polish as well. Previously, I extended and
built upon the original code, but I can't give it awa
Thanks for the tip.
One small improvement on the IndexAccessorFactory might be to allow user
to specify the Analyzer instead of using a default KeywordAnalyzer,
which of course will make your static init of the cached accessors
difficult unless you add more interfaces to the accessor to allow r
One final noteif you are using the IndexAccessor and you are only
accessing the index from one JVM, you can use the NoLockFactory and save
some sync cost there.
Jay Yu wrote:
Mark,
Great effort getting the original lucene index accessor package in
this shape. I am sure this will benefit
2.3-dev is not required. I'd have to look and see what is...I'd guess
around 2.1. I did program for java 1.5 though, but converting to 1.4
should be minimal work.
Jay Yu wrote:
Mark,
Great effort getting the original lucene index accessor package in
this shape. I am sure this will benefit a
Mark,
Great effort getting the original lucene index accessor package in this
shape. I am sure this will benefit a lot of people using Lucene in a
multithread env.
I have a quick question to ask you:
Do you have to use the core Lucene 2.3-dev in order to use the accessor?
I will take a look a
I have sat down and rewrote IndexAccessor from scratch. I copied in the
same reference counting logic, pruned some things, and tried to make the
whole package a bit simpler to use. I have a few things to do, but its
pretty solid already. The only major thing I'd still like to do is add
an optio
I'd be very interested to see your test results and codes. Thanks!
Mark Miller wrote:
I sat down over the weekend and rewrote the code from scratch so that I
could improve and simplify it somewhat. I also did some testing of the
synch costs, and it is very insignificant compared to the total ti
I sat down over the weekend and rewrote the code from scratch so that I
could improve and simplify it somewhat. I also did some testing of the
synch costs, and it is very insignificant compared to the total time to
parse a query and run a search. I'll try and get around to posting the
code toni
Mark Miller wrote:
Good luck Jay. Keep in mind, pretty much all LuceneIndexAccessor does is
sync Readers with Writers and allow multiple threads to share the same
instances of them -- nothing more. The code just forces Readers to
refresh when Writers are used to change the index. There really
Good luck Jay. Keep in mind, pretty much all LuceneIndexAccessor does is
sync Readers with Writers and allow multiple threads to share the same
instances of them -- nothing more. The code just forces Readers to
refresh when Writers are used to change the index. There really isn't
any functional
Mark,
Thanks for sharing your valuable exp. and thoughts.
Frankly our system already has most of the functionalities
LuceneIndexAcessor offers. The only thing I am looking for is to sync
the searchers' close. That's why I am little worried about the way
accessor handles the searcher sync.
I w
The method is synched, but this is because each thread *does* share the
same Searcher. To maintain a cache of searchers across multiple threads,
you've got to sync -- to reference count, you've got to sync. The
performance hit of LuceneIndexAcessor is pretty minimal for its
functionality, and f
Mark,
After reading the implementation of LuceneIndexAccessor.getSearcher(),
I realized that the method is synchronized and wait for writingDirector
to be released. That means if we getSearcher for each query in each
thread, there might be a contention and performance hit. In fact, even
the me
Thanks for your detailed explanation of the issues and your solutions.
It seems that LuceneIndexAccessor is worth trying first before I
implement other locking mechanism to ensure proper order.
I will appreciate it very much if you'd like your extension with us.
Jay
Mark Miller wrote:
Ill res
Ill respond a point at a time:
1.
** Hi Maik,
So what happens in this case:
IndexAccessProvider accessProvider = new IndexAccessProvider(directory,
analyzer);
LuceneIndexAccessor accessor = new LuceneIndexAccessor(accessProvider);
accessor.open();
IndexWriter wri
Mark,
thanks for sharing your insight and experience about LuceneIndexAccessor!
I remember seeing some people reporting some issues about it, such as:
http://www.archivum.info/[EMAIL PROTECTED]/2005-05/msg00114.html
http://issues.apache.org/bugzilla/show_bug.cgi?id=34995#c3
Have those issues bee
I use option 3 extensivley and find it very effective. There is a tweak or
two required to get it to work right with MultiSearchers, but other than
that, the code is great. I have built a lot on top of it. I'm on the list
all the time and would be happy to answer any questions you have in regards
t
In a multithread app like web app, a shared IndexSearcher could throw a
AlreadyClosedException when another thread is trying to update the
underlying IndexReader by closing the shared searcher after the index is
updated. Searching over the past discussions on this mailing list, I
found several
22 matches
Mail list logo
