Re: Synchronous Lucene index update tests occasionally fail

t;> > >>> Can casing issue appear randomly? Because tests not fail constantly, > >>> they > >>> are > >>> blinking. > >>> > >>> Daniel. > >>> > >>> > >>> > >>> -- > >>

Re: Synchronous Lucene index update tests occasionally fail

>> > wrote: >> >>> Can casing issue appear randomly? Because tests not fail constantly, >>> they >>> are >>> blinking. >>> >>> Daniel. >>> >>> >>> &

Re: Synchronous Lucene index update tests occasionally fail

Tue, Jun 28, 2016 at 4:04 AM, Daniq < > gdaniq@ > > wrote: > >> Can casing issue appear randomly? Because tests not fail constantly, they >> are >> blinking. >> >> Daniel. >> >> >> >> -- >> View this message in context: >&

Re: Synchronous Lucene index update tests occasionally fail

linking. > > Daniel. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Synchronous-Lucene-index-update-tests-occasionally-fail-tp4283970p4284657.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > --

Re: Synchronous Lucene index update tests occasionally fail

Can casing issue appear randomly? Because tests not fail constantly, they are blinking. Daniel. -- View this message in context: http://lucene.472066.n3.nabble.com/Synchronous-Lucene-index-update-tests-occasionally-fail-tp4283970p4284657.html Sent from the Lucene - Java Users mailing list

Re: Synchronous Lucene index update tests occasionally fail

It looks like you have a casing issue maybe? You indexed variableWithHelpString. But searched for variablewithhelpstring* Mike McCandless http://blog.mikemccandless.com On Mon, Jun 27, 2016 at 5:07 AM, Baskakov Daniel wrote: > I've just noticed that not only dynamic adding/removing entities

Re: Synchronous Lucene index update tests occasionally fail

I've just noticed that not only dynamic adding/removing entities tests fail, but also a simple indexing. Here is a boiled down structure of the test: @BeforeClass public static void beforeClass() throws Exception { // ContextManager is a domain model contextManager = createContextMa

Re: Synchronous Lucene index update tests occasionally fail

Can you boil this down to a small standalone test case showing the issue? Mike McCandless http://blog.mikemccandless.com On Mon, Jun 27, 2016 at 4:03 AM, Baskakov Daniel wrote: > Thank you Mike. > > Commit is performed after each indexing op in unit tests only: > > public void commitNow() th

Re: Synchronous Lucene index update tests occasionally fail

Thank you Mike. Commit is performed after each indexing op in unit tests only: public void commitNow() throws IOException { if (getIndexWriter().hasUncommittedChanges()) { getIndexWriter().commit(); } } In production environment I have a timer that performs commit periodi

Re: Synchronous Lucene index update tests occasionally fail

You must reopen your IndexReader to see recent changes to the index. But, IW.commit after each indexing op is very costly. It's much better to get near-real-time readers, e.g. from a SearcherManager that you pass your IW instance too, after each set of changes that you now need to search. As lon

Synchronous Lucene index update tests occasionally fail

Originally i've posted the question at stackoverflow.com but without any reply. So I hope someone can help me in the official list. I'm testing that dynamic changes of the domain model reflects at the Lucene index. Special event listeners (synchronous, no multithreading here) are executed when the

Re: Is housekeeping of Lucene indexes block index update but allow search ?

if most searches do not need to wait for a specific index generation. private final ControlledRealTimeReopenThread indexSearcherReopenThread; private long reopenToken; // index update/delete methods returned token private static final Log log = LogFactory.getLog(LuceneEngineInstance.class

Re: Is housekeeping of Lucene indexes block index update but allow search ?

Hi Gaurav Thanks for the clarification. If possible, please share your NRT manager API related code example. i believe, it will help me to understand little better. - Kumaran R On Tue, Aug 5, 2014 at 12:39 PM, Gaurav gupta wrote: > Thanks Kumaran and Erik for resolving my queries. > > K

Re: Is housekeeping of Lucene indexes block index update but allow search ?

Thanks Kumaran and Erik for resolving my queries. Kumaran, You are right at only one indexwriter can write as it acquire the lock but using the NRT manager APis - TrackingIndexWriter multiple concurrent upd

Re: Is housekeeping of Lucene indexes block index update but allow search ?

Right. 1> Occasionally the merge will require 2x the disk space. (3x in compound file system). The merging is, indeed, done in the background, it is NOT a blocking operation. 2> n/a. It shouldn't block at all. Here's a cool video by Mike McCandless on the merging process, plus some explanations:

Re: Is housekeeping of Lucene indexes block index update but allow search ?

Hi Gaurav 1.When you opened index to write,till you close that index, there will be a lock to do further write. But not for search. During merge, index needs 3X ( not sure 2X?) of more storage space, i believe that is the reason for no blocking for search. ( any other experts can clarify you more

Is housekeeping of Lucene indexes block index update but allow search ?

Hi, We are planning to use Lucene 4.8.1 over Oracle (1 to 2 TB data) and seeking information on "How Lucene conduct housekeeping or maintenance of indexes over a period of time". *Is it a blocking operation for write and search or it will not block anything while merging is going on? * I found :

Re: Lucene index update

One major reason is to update a field or rather shadow a field. i have a field named "testField" in index1 and i want to update that field. When I update, I want only the new value to be reflected, not the value in old field. now parallelreader starts from the latest index, i.e index2 and searches

Re: Lucene index update

But why do you feel the need to have a parallel reader that combines result sets across two indices based on docId? On Thu, Oct 28, 2010 at 12:17 AM, Nilesh Vijaywargiay < nilesh.vi...@gmail.com> wrote: > Pulkit, > Parallel reader takes the union of all fields for a given id. Thus if I > want > t

Re: Lucene index update

Pulkit, Parallel reader takes the union of all fields for a given id. Thus if I want to add a field or modify a field of a document which has id 2 in index1, I need to createa a document with id 2 in index2 with the fields I want to add/modify. Thus parallel reader would treat them as fields of a s

Re: Lucene index update

Look interesting, what is the merit in having a second index in order to keep the document id the same? Perhaps I have misunderstood. Just want to understand your motivation here. On Wed, Oct 20, 2010 at 2:57 PM, Nilesh Vijaywargiay wrote: > I've written a blog regarding a work around for updati

Lucene index update

I've written a blog regarding a work around for updating index in Lucene using parallel reader. It's explained with results and pictures. It would be great if you have a look at it. The link: http://the10minutes.blogspot.com/2010/10/lucene-index-update.html

Re: Index Update

> IndexReader on the search side, nothing happens on the search side. > >> > > > >> > > Copying like you do is absolutely not needed. > >> > > > >> > > - > >> > > Uwe Schindler > >> > > H.-H.-Meier-Allee

Re: Index Update

do is absolutely not needed. >> > > >> > > - >> > > Uwe Schindler >> > > H.-H.-Meier-Allee 63, D-28213 Bremen >> > > http://www.thetaphi.de >> > > eMail: u...@thetaphi.de >> > > >> > > > -Orig

Re: Index Update

t; > > > - > > > Uwe Schindler > > > H.-H.-Meier-Allee 63, D-28213 Bremen > > > http://www.thetaphi.de > > > eMail: u...@thetaphi.de > > > > > > > -Original Message- > > > > From: Weiwei Wang [mailto:ww.wang...@gma

Re: Index Update

hi.de > > eMail: u...@thetaphi.de > > > > > -Original Message- > > > From: Weiwei Wang [mailto:ww.wang...@gmail.com] > > > Sent: Saturday, December 12, 2009 1:25 PM > > > To: java-user@lucene.apache.org > > > Subject: Index Update

Re: Index Update

-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > -Original Message- > > From: Weiwei Wang [mailto:ww.wang...@gmail.com] > > Sent: Saturday, December 12, 2009 1:25 PM > > To: java-user@lucene.apache.org > > Subj

RE: Index Update

> -Original Message- > From: Weiwei Wang [mailto:ww.wang...@gmail.com] > Sent: Saturday, December 12, 2009 1:25 PM > To: java-user@lucene.apache.org > Subject: Index Update > > Hi, all, > I need to update my index everyday so as to keep pace with times. > Current

Index Update

Hi, all, I need to update my index everyday so as to keep pace with times. Current I have two copy of indexes, one is used to provide search service, the other is used to do updating. Once the updating process is done, I write all the updated documents to the index which provide search service

Re: index update problems with Linux

Kevin, I don't see writer.close() in your code snippet. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Kevin Dewi <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Friday, January 18, 2008 6:33:43 AM Subject: index upda

Re: index update problems with Linux

it seems that close reader not release write lock, test lock release before you add document 2008/1/18, Kevin Dewi <[EMAIL PROTECTED]>: > > Hello, > > I have a problem with this code (updating a lucene index by delete and > adding documents): > > > IndexReader reader = IndexReader.open(directory)

index update problems with Linux

Hello, I have a problem with this code (updating a lucene index by delete and adding documents): IndexReader reader = IndexReader.open(directory); while (i.hasNext()) { reader.deleteDocuments(i.next()); } reader.close(); ... IndexWriter writer = new IndexWriter(directory,

Re: Question regarding Index Update

Then I think you'll be disappointed. Search the archive for phrases like "update in place" and you'll see a discussion of why this isn't as straight-forward as you might think. Best Erick On 7/9/07, Sonu SR <[EMAIL PROTECTED]> wrote: Thanks Erik. Is the field update is too difficult task in lu

Re: Question regarding Index Update

Thanks Erik. Is the field update is too difficult task in lucene? I expect this feature also in lucene, in near future. On 7/9/07, Erick Erickson <[EMAIL PROTECTED]> wrote: No, Lucene doesn't support just updating a field in a document. You must delete/re-add it as you suppose. And you are cor

Re: Question regarding Index Update

No, Lucene doesn't support just updating a field in a document. You must delete/re-add it as you suppose. And you are correct, the UpdateDocument feature just conceals the underlying delete/add functionality, it's not update-in-place. Best Erick On 7/9/07, Sonu SR <[EMAIL PROTECTED]> wrote: H

Question regarding Index Update

Hi, My application using lucene index. The index documents having number of fields. We have around 5 million such documents. I have problem with regular update of some of the fields in the document. So every update I need to delete and index that updated documents. This is a huge task. I know from

Re: Index Update Strategies

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . http://www.lucene-consulting.com/ - Original Message From: Andreas Guther <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Wednesday, April 25, 2007 3:09:29 PM Subject: Index Update Strategies Hi We have an index of seve

Index Update Strategies

Hi We have an index of several GB in size which is updated very frequently-about every 2 seconds. Though it is desired to have changes updated to the index as soon as possible I wonder if this frequent updates can have negative affect on the search and data retrieval performance. Would it make m

remote index update question

Hello, I have 5 remote indexes being updated using the following methods: 1. one machine indexes documents and creates lucene doc objects. 2. lucene docs are batched, put into maps 3. maps are serialized and posted over http to servlets on each remote machine. 4. receiving servlets get maps, pul

Re: Lucene index update

Ivan - Original Message - From: "Erick Erickson" <[EMAIL PROTECTED]> To: Sent: Thursday, January 04, 2007 5:40 PM Subject: Re: Lucene index update > for approach <2>, I *think* you can extract information about unstored > data > by playing with TermDocs/TermEn

Re: Lucene index update

04, 2007 5:40 PM Subject: Re: Lucene index update for approach <2>, I *think* you can extract information about unstored data by playing with TermDocs/TermEnums. Conceptually, the idea is to go through all the terms and, for document (lucene ID) 1 find the terms that appear in docume

Re: Lucene index update

for approach <2>, I *think* you can extract information about unstored data by playing with TermDocs/TermEnums. Conceptually, the idea is to go through all the terms and, for document (lucene ID) 1 find the terms that appear in document 1 and order them by their termpositions. Repeat for document

Lucene index update

Hi All, I want to update some documents in existing indexes by adding a new field to each of their documents. The documents contained in the indexes have some fields that are indexed and NOT stored. The new field that will be added will contain some metadata and will be Stored and not indexe

Re: index update with database insertion

better still (for transaction isolation, but arguably not performance) do the index update within the same transaction. I have just been implementing some test code using Compass (Open Symphony) which provides full transaction isolation for Lucene. That is an excellent point Jason -- that thread

Re: index update with database insertion

transaction isolation, but arguably not performance) do the index update within the same transaction. I have just been implementing some test code using Compass (Open Symphony) which provides full transaction isolation for Lucene. On 8/21/06, Michael McCandless <[EMAIL PROTECTED]> wrote: >

Re: index update with database insertion

> In my project,I want to update the lucene's index when there has database > insertion operations,in this way,my users could search the fresh information > immediately if someone inserted the information into database.That's what I > need,could someone give me suggestions to implement my need?