can you provide your query which yields all the documents that you want to delete? I don't understand how the sort order changes anything here. if you want to only delete the top N docs of that query you should maybe modify your query to only return those. I could imagine you are returning the oldest first, if so can't you do a range filter on top instead of sorting?
simon On Wed, Sep 14, 2011 at 5:06 PM, Ian Lea <ian....@gmail.com> wrote: > Could never have told that from what you wrote ... > > Anyway, I've no idea, beyond 2 general statements: > > 1. Lucene is amazingly fast. > 2. Doing anything on large numbers of docs is liable to take some time. > > Make of that what you will. If it was me, I'd test it. > > > -- > Ian. > > > On Wed, Sep 14, 2011 at 3:28 PM, <v.se...@lombardodier.com> wrote: >> Hi, >> >> this was clear actually. I was questionning the performance impact to call >> IndexReader.deleteDocument(int docNum) one million time. any information >> about that? >> >> thanks, >> vincent >> >> >> >> >> >> >> >> Ian Lea <ian....@gmail.com> >> >> >> 14.09.2011 16:20 >> Please respond to >> java-user@lucene.apache.org >> >> >> >> To >> java-user@lucene.apache.org >> cc >> >> Subject >> Re: deleting with sorting and max document >> >> >> >> >> >> >> You don't do it with deleteDocuments() but with >> IndexReader.deleteDocument(int docNum), as I said. >> >> To spell it out a bit more: >> >> Execute a search to get a list of document ids, >> searching/sorting/whatever as you wish. >> >> Then loop through that list of document ids calling >> IndexReader.deleteDocument(int docNum) for each one. >> >> >> Hope that's clearer. >> >> >> -- >> Ian. >> >> >> On Wed, Sep 14, 2011 at 3:13 PM, <v.se...@lombardodier.com> wrote: >>> Hi, thanks for your answer. out of the 35 millions docs, I need to >> delete >>> 1 million... >>> >>> and unfortunately, the ability to put a sort and a max event is not on >> the >>> query, but as args in the index searcher. >>> >>> so I do not see how to do it with deleteDocuments. >>> >>> regards, >>> >>> vincent >>> >>> >>> >>> >>> >>> >>> >>> Ian Lea <ian....@gmail.com> >>> >>> >>> 14.09.2011 15:37 >>> Please respond to >>> java-user@lucene.apache.org >>> >>> >>> >>> To >>> java-user@lucene.apache.org >>> cc >>> >>> Subject >>> Re: deleting with sorting and max document >>> >>> >>> >>> >>> >>> >>> You can get a list of document ids via your search/sort call and call >>> IndexReader.deleteDocument(int docNum) for each one. >>> >>> >>> -- >>> Ian. >>> >>> >>> On Wed, Sep 14, 2011 at 2:23 PM, <v.se...@lombardodier.com> wrote: >>>> Hi, >>>> >>>> I have an index with 35 millions docs in it. every day I need to delete >>>> some of the oldest docs that meet some criteria. >>>> >>>> I can easily do this on the searcher by using search(Query query, int >> n, >>>> Sort sort) >>>> >>>> but there is nothing equivalent for the deleteDocuments. >>>> >>>> what are my options? >>>> >>>> thanks, >>>> >>>> vincent >>>> >>>> ************************ DISCLAIMER ************************ >>>> This message is intended only for use by the person to >>>> whom it is addressed. It may contain information that is >>>> privileged and confidential. Its content does not >>>> constitute a formal commitment by Lombard Odier >>>> Darier Hentsch & Cie or any of its branches or affiliates. >>>> If you are not the intended recipient of this message, >>>> kindly notify the sender immediately and destroy this >>>> message. Thank You. >>>> ***************************************************************** >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>> >>> >>> >>> >>> ************************ DISCLAIMER ************************ >>> This message is intended only for use by the person to >>> whom it is addressed. It may contain information that is >>> privileged and confidential. Its content does not >>> constitute a formal commitment by Lombard Odier >>> Darier Hentsch & Cie or any of its branches or affiliates. >>> If you are not the intended recipient of this message, >>> kindly notify the sender immediately and destroy this >>> message. Thank You. >>> ***************************************************************** >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >> >> >> ************************ DISCLAIMER ************************ >> This message is intended only for use by the person to >> whom it is addressed. It may contain information that is >> privileged and confidential. Its content does not >> constitute a formal commitment by Lombard Odier >> Darier Hentsch & Cie or any of its branches or affiliates. >> If you are not the intended recipient of this message, >> kindly notify the sender immediately and destroy this >> message. Thank You. >> ***************************************************************** >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
