Re: Why read past EOF

2012-02-17 Thread Michael McCandless
Hmm, though, one question: if you are using a single IndexWriter, always on the same machine, then it should not matter that the computer's clock is way off. Because, the DeletionPolicy is comparing timestamps pulled only from a single clock. Ie the shift won't matter; only relative comparisons m

Re: Why read past EOF

2012-02-17 Thread Michael McCandless
OK, thanks for bringing closure! Mike McCandless http://blog.mikemccandless.com On Thu, Feb 16, 2012 at 10:08 PM, superruiye wrote: > Oh,I made a mistake.Our testing server's time is faster hours than it should > be.I reminded workmate to modify it,and index maintain in a range size. > Thank y

Re: Why read past EOF

2012-02-16 Thread superruiye
Oh,I made a mistake.Our testing server's time is faster hours than it should be.I reminded workmate to modify it,and index maintain in a range size. Thank you all the same. -- View this message in context: http://lucene.472066.n3.nabble.com/Why-read-past-EOF-tp3639401p3752592.html Sent from the L

Re: Why read past EOF

2012-02-16 Thread Michael McCandless
Wait: I see your DP above calling .delete() -- can you verify that code is in fact invoked? EG print on each onCommit how many commits are deleted and how many are not? Mike McCandless http://blog.mikemccandless.com On Wed, Feb 15, 2012 at 9:21 PM, superruiye wrote: > My IndexWriter only creat

Re: Why read past EOF

2012-02-15 Thread superruiye
My IndexWriter only create once and cached in memery. I restart tomcat this morning,and the index become 94M...But I restart yesterday serveral times ,it still too big... My deletion policy is in above reply,it only compare the timestamp,not actually deleting commits. -- View this message in cont

Re: Why read past EOF

2012-02-15 Thread Michael McCandless
Is your deletion policy actually deleting commits? Mike McCandless http://blog.mikemccandless.com On Wed, Feb 15, 2012 at 5:21 AM, superruiye wrote: > http://lucene.472066.n3.nabble.com/file/n3746464/index.jpg > > The index files are same size,and the index increase to 7.5G in one day,but > it

Re: Why read past EOF

2012-02-15 Thread superruiye
http://lucene.472066.n3.nabble.com/file/n3746464/index.jpg The index files are same size,and the index increase to 7.5G in one day,but it should only 90-100M... -- View this message in context: http://lucene.472066.n3.nabble.com/Why-read-past-EOF-tp3639401p3746464.html Sent from the Lucene - J

Re: Why read past EOF

2012-02-11 Thread Michael McCandless
I'm glad the timed deletion policy is working on NFS! Thanks for bringing closure, Mike McCandless http://blog.mikemccandless.com On Fri, Feb 10, 2012 at 9:58 PM, superruiye wrote: > Thanks for your advice and patient. > I modify "present",and use stress testing two day(loop search and index),

Re: Why read past EOF

2012-02-10 Thread superruiye
Thanks for your advice and patient. I modify "present",and use stress testing two day(loop search and index),the "read past EOF" didn't appeared yet. My system structure should be improved, I hope to get your proposal again in the future. -- View this message in context: http://lucene.472066.n3.n

Re: Why read past EOF

2012-02-08 Thread Michael McCandless
Hmm, there's a problem with the logic here (sorry: this is my fault -- my prior suggestion is flat out wrong!). The problem is... say you commit once, creating commit point 1. Two hours later, you commit again, creating commit point 2. The bug is, at this point, immediately on committing commit

Re: Why read past EOF

2012-02-07 Thread superruiye
public class PostponeCommitDeletionPolicy implements IndexDeletionPolicy { private final static long deletionPostPone = 60; public void onInit(List commits) { // Note that commits.size() should normally be 1: onCommit(commits); }

Re: Why read past EOF

2012-02-06 Thread superruiye
ok,thanks. I modify my program like you suggest.But another problem appear: java.lang.ArrayIndexOutOfBoundsException: -1 at org.apache.lucene.index.TermInfosReader.seekEnum(TermInfosReader.java:203) at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:273) at

Re: Why read past EOF

2012-02-03 Thread Michael McCandless
Instead of .getVersion() you should use .getTimestamp()... version is not "really" a timestamp. (Though, really, you should store your own timestamp inside the commit userData, and retrieve that, instead... the getTimestamp API will be deprecated in 3.6.0). Also, you may need to implement onInit,

Re: Why read past EOF

2012-02-02 Thread superruiye
eg,I implement IndexDeletionPolicy and the onCommit(): public void onCommit(List commits) { // Note that commits.size() should normally be 2 (if not // called by onInit above): int size = commits.size(); long present = System.

Re: Why read past EOF

2012-02-02 Thread superruiye
Thanks,you suggest me to creat a my IndexDeletionPolicy,I check KeepOnlyLastCommitDeletionPolicy.onCommit,it invoke CommitPoint.delete(),but it only : deleted = true; commitsToDelete.add(this); ,and the delete index file operation in other place. Is it useful that I postpone invoke

Re: Why read past EOF

2012-02-01 Thread Michael McCandless
Right, you have to ensure (by using the "right" IndexDeletionPolicy) that no commit is ever removed until all readers open against that commit have been closed. "Normally" the filesystem ensures this for us (protects still-open files from being deleted), but NFS (unfortunately!) lacks such semanti

Re: Why read past EOF

2012-01-31 Thread superruiye
Does it means I only to ensure reopen readers before deleted.I use default IndexDeletionPolicy: KeepOnlyLastCommitDeletionPolicy.And another two IndexDeletionPolicy,SnapshotDeletionPolicy and PersistentSnapshotDeletionPolicy,I am watching now.Are they useful to this problem? -- View this message i

Re: Why read past EOF

2012-01-09 Thread Michael McCandless
OK then that's the problem. Unlike local file systems, NFS makes no effort to protect still-open-for-read files from being deleted (which Lucene by default relies on by default). The solution is easy: create your own IndexDeletionPolicy to "protect" old index commit points from being deleted unti

Re: Why read past EOF

2012-01-09 Thread superruiye
yes，i use it to share index,and one server is master to writer and read;the others are slaver,read only. -- View this message in context: http://lucene.472066.n3.nabble.com/Why-read-past-EOF-tp3639401p3644074.html Sent from the Lucene - Java Users mailing list archive at Nabble.com.

Re: Why read past EOF

2012-01-07 Thread Michael McCandless
Is the index accessed over NFS? Mike McCandless http://blog.mikemccandless.com On Fri, Jan 6, 2012 at 9:28 PM, superruiye wrote: > Hi, >   I use lucene 3.4.0 in a search project,but encounter a problem and i > don't know how to resolve. > I index and it run well,but one week or two(it appear tw

Re: Why read past EOF

2012-01-07 Thread superruiye
and my index create is: IndexWriterConfig indexWriterConfig = new IndexWriterConfig(Version.LUCENE_34, getAnalyzer()); indexWriterConfig.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND); writer = new IndexWriter(directory, indexWriterConfig); writer.commit(); -- View this message in conte

Re: Why read past EOF

2012-01-07 Thread superruiye
and my IndexWriter create is: IndexWriterConfig indexWriterConfig = new IndexWriterConfig(Version.LUCENE_34, getAnalyzer()); indexWriterConfig.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND); writer = new IndexWriter(directory, indexWriterConfig); writer.commit(); -- View this message i