Re: Remoting Lucene

2012-01-09 Thread findbestopensource
Hi, One good option is to consider using Solr as it helps to access the index remotely. If you want to use Lucene and you are ready to build your own API then you could have a web application, which will receive user query, search in the index and return the result set in user expected fashion. Y

Remoting Lucene

2012-01-09 Thread Trejkaz
Hi all. I want to access a Lucene index remotely. I'm aware of a couple of options for it which seem to operate more or less at the IndexSearcher level - send a query, get back results. But in our case, we use IndexReader directly for building statistics, which is too slow to do via individual q

Re: Tamper resistant index

2012-01-09 Thread Andrzej Bialecki
On 09/01/2012 16:27, Mike C wrote: Hi, I'm investigating storing syslog data using Lucene (via Solr or Elasticsearch, undecided at present). The syslogs belong to systems under the scope of the PCI DSS (Data Security Standard), and one of the requirements is to ensure logs aren't tampered with.

Re: Help running out of files

2012-01-09 Thread Charlie Hubbard
Ian, Thanks for you help and patients with me. I started to look at porting this to a simple self contained example, and I finally found the error in my code. IndexSearcher.close() doesn't close the underlying IndexReader when using the constructor new IndexSearcher( IndexReader ). Went back an

Re: Help running out of files

2012-01-09 Thread Ian Lea
Charlie >From the FAQ >http://wiki.apache.org/lucene-java/LuceneFAQ#Does_Lucene_allow_searching_and_indexing_simultaneously.3F "... an IndexReader only searches the index as of the "point in time" that it was opened. Any updates to the index, either added or deleted documents, will not be visi

Tamper resistant index

2012-01-09 Thread Mike C
Hi, I'm investigating storing syslog data using Lucene (via Solr or Elasticsearch, undecided at present). The syslogs belong to systems under the scope of the PCI DSS (Data Security Standard), and one of the requirements is to ensure logs aren't tampered with. I'm looking for advice on how to acco

Re: Help running out of files

2012-01-09 Thread Charlie Hubbard
Ian, >From reading the docs it's seems clear all I need to do is call IndexWriter.commit() in order for the changes to my single IndexWriter to be visible to the IndexReader and hence my single IndexSearcher. When you say "you need to close old readers, you need to reopen readers to pick up chang

Re: Why read past EOF

2012-01-09 Thread Michael McCandless
OK then that's the problem. Unlike local file systems, NFS makes no effort to protect still-open-for-read files from being deleted (which Lucene by default relies on by default). The solution is easy: create your own IndexDeletionPolicy to "protect" old index commit points from being deleted unti

Re: Using dismax features in Lucene

2012-01-09 Thread Ian Lea
Ah, OK, showing my ignorance again. I don't use Solr and haven't read the book. FWIW I'm not aware of anything else related already present in core lucene. But DisjunctionMaxQuery is just a query so could be combined with other queries, such as PhraseQuery, much as you're already doing by the so

Re: Using dismax features in Lucene

2012-01-09 Thread Paul Taylor
On 09/01/2012 10:50, Ian Lea wrote: Use the oal.search.DisjunctionMaxQuery class present in core lucene? -- Ian. The book said that dismax query was similar but different to DisjunctionMaxQuery and additionally did Phrase Boosting which I didnt think DisjunctionMaxQuery did. On Fri, J

3.5.0 javadocs link missing?

2012-01-09 Thread Ian Lea
Hi The "Documentation" link on http://lucene.apache.org/java/docs/index.html expands to list Release 3.4.0, 3.3.0, etc. but not 3.5.0. http://lucene.apache.org/java/3_5_0/ exists and works. -- Ian. - To unsubscribe, e-mail: j

Re: Using dismax features in Lucene

2012-01-09 Thread Ian Lea
Use the oal.search.DisjunctionMaxQuery class present in core lucene? -- Ian. On Fri, Jan 6, 2012 at 10:52 PM, Paul Taylor wrote: > Just reading Apache Solr Enterprise Search Server and was interested in > pages 152, 153 dismax and DisjunctionMaxQuery and automatic Phrase Boosting. > I would li

Re: How to merge indices in ram

2012-01-09 Thread Ian Lea
Buy better hardware? Some tasty stuff mentioned last week. There have been reports of spectacular lucene performance on SSDs. -- Ian. On Mon, Jan 9, 2012 at 7:06 AM, dyzc2010 wrote: > Hi, > > > How can I merge multiple indices in RAM while not impacting search? > > > Thanks

Re: How to load only part of index files on hard drive to ram

2012-01-09 Thread Ian Lea
You could certainly run some warm up queries, but you can't tell lucene which bits of an index to load into RAM. There are a couple of classes in the org.apache.lucene.index contrib section that can split indexes but I doubt they are relevant here. Why the obsession with RAMDirectory? In my exp

Re: Build RAMDirectory on FSDirectory, and then synchronzing the two

2012-01-09 Thread Ian Lea
If you load an existing disk index into a RAMDirectory, make some changes in RAM and call addIndexes to add the contents of the RAMDirectory to the original disk index, you are likely to end up with duplicate data on disk. Depending of course on what you've done to the RAM index. Sounds you want

Re: Help running out of files

2012-01-09 Thread Ian Lea
It's hard, impossible for me, to figure out from this what your problem might be, Multiple indexes, MultiReader, multiple writers (?), multiple threads? However I can make some statements: Lucene doesn't leak files, you need to close old readers, you need to reopen readers to pick up changes. Ha

Re: Why read past EOF

2012-01-09 Thread superruiye
yes，i use it to share index,and one server is master to writer and read;the others are slaver,read only. -- View this message in context: http://lucene.472066.n3.nabble.com/Why-read-past-EOF-tp3639401p3644074.html Sent from the Lucene - Java Users mailing list archive at Nabble.com.