Hi Christoph and Shai, Thanks for the quick response!. Indices are stored in a relational database ( using a custom Directory implementation ). The Problem comes since the indices are sharded (both taxonomy indices and normal doc indices), when a user wants to drilldown, I have to merge all the indices. For that I used mergeUtils (which worksperfect). For now I am using RAMDirectory as the merged indices. Anyway The indices can grow to a bigger size as time goes. MMapDirectory again uses memory right? Can It deal with possible out of memory issue?
I am thinking of using the same Database to store the merged indices. But the problem is the original sharded indices can be updated, when new entries come in. So the merged final indices also needs to be updated accordingly. On Thu, Apr 2, 2015 at 4:55 PM, Shai Erera <ser...@gmail.com> wrote: > In some cases, MMapDirectory offers even better performance, since the JVM > doesn't need to manage that RAM when it's doing GC. > > Also, using only RAMDirectory is not safe in that if the JVM crashes, your > index is lost. > > On Thu, Apr 2, 2015 at 12:54 PM, Christoph Kaser <lucene_l...@iconparc.de> > wrote: > > > Hi Gimantha, > > > > why do you use a RAMDirectory? If your merged index fits into RAM > > completely, a MMapDirectory should offer almost the same performance. And > > if not, it is definitely the better choice. > > > > Regards > > Christoph > > > > > > Am 02.04.2015 um 12:38 schrieb Gimantha Bandara: > > > >> Hi All, > >> > >> I have successfully setup a merged indices and drilldown and usual > search > >> operations work perfect. > >> But, I have a side question. If I selected RAMDirectory as the > destination > >> Indices in merging, probably the jvm can go out of memory if the merged > >> indices are too big. Is there a way I can handle this issue? > >> > >> On Tue, Mar 24, 2015 at 12:18 PM, Gimantha Bandara <giman...@wso2.com> > >> wrote: > >> > >> Hi Christoph, > >>> > >>> My mistake. :) It does the exactly what i need. figured it out later.. > >>> Thanks a lot! > >>> > >>> On Tue, Mar 24, 2015 at 3:14 AM, Gimantha Bandara <giman...@wso2.com> > >>> wrote: > >>> > >>> Hi Christoph, > >>>> > >>>> I think TaxonomyMergeUtils is to merge a taxonomy directory and an > index > >>>> together (Correct me if I am wrong). Can it be used to merge several > >>>> taxonomyDirectories together and create one taxonomy index? > >>>> > >>>> On Mon, Mar 23, 2015 at 9:19 PM, Christoph Kaser < > >>>> lucene_l...@iconparc.de > >>>> > >>>>> wrote: > >>>>> Hi Gimantha, > >>>>> > >>>>> have a look at the class org.apache.lucene.facet. > >>>>> taxonomy.TaxonomyMergeUtils, > >>>>> which does exactly what you need. > >>>>> > >>>>> Best regards, > >>>>> Christoph > >>>>> > >>>>> Am 23.03.2015 um 15:44 schrieb Gimantha Bandara: > >>>>> > >>>>> Hi all, > >>>>>> > >>>>>> Can anyone point me how to merge several taxonomy indexes? My > >>>>>> requirement > >>>>>> is as follows. I have several taxonomy indexes and normal document > >>>>>> indexes. I want to merge taxonomy indexes together and other > document > >>>>>> indexes together and perform search on them. One part I have figured > >>>>>> out. > >>>>>> It is easy. To Merge document indexes, all I have to do is create a > >>>>>> MultiReader and pass it to IndexSearcher. But I am stuck at merging > >>>>>> the > >>>>>> taxonomy indexes. Is there a way to merge taxonomy indexes? > >>>>>> > >>>>>> > >>>>>> -- > >>>>> Dipl.-Inf. Christoph Kaser > >>>>> > >>>>> IconParc GmbH > >>>>> Sophienstrasse 1 > >>>>> 80333 München > >>>>> > >>>>> www.iconparc.de > >>>>> > >>>>> Tel +49 -89- 15 90 06 - 21 > >>>>> Fax +49 -89- 15 90 06 - 49 > >>>>> > >>>>> Geschäftsleitung: Dipl.-Ing. Roland Brückner, Dipl.-Inf. Sven > Angerer. > >>>>> HRB > >>>>> 121830, Amtsgericht München > >>>>> > >>>>> > >>>>> > >>>>> --------------------------------------------------------------------- > >>>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > >>>>> For additional commands, e-mail: java-user-h...@lucene.apache.org > >>>>> > >>>>> > >>>>> -- > >>>> Gimantha Bandara > >>>> Software Engineer > >>>> WSO2. Inc : http://wso2.com > >>>> Mobile : +94714961919 > >>>> > >>>> -- > >>> Gimantha Bandara > >>> Software Engineer > >>> WSO2. Inc : http://wso2.com > >>> Mobile : +94714961919 > >>> > >>> > >> > > > > -- > > Dipl.-Inf. Christoph Kaser > > > > IconParc GmbH > > Sophienstrasse 1 > > 80333 München > > > > www.iconparc.de > > > > Tel +49 -89- 15 90 06 - 21 > > Fax +49 -89- 15 90 06 - 49 > > > > Geschäftsleitung: Dipl.-Ing. Roland Brückner, Dipl.-Inf. Sven Angerer. > HRB > > 121830, Amtsgericht München > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919
