Btw I was using a RAMDirectory for just testing purposes.. On Thu, Apr 2, 2015 at 5:16 PM, Gimantha Bandara <giman...@wso2.com> wrote:
> Hi Christoph and Shai, > > Thanks for the quick response!. > Indices are stored in a relational database ( using a custom Directory > implementation ). The Problem comes since the indices are sharded (both > taxonomy indices and normal doc indices), when a user wants to drilldown, I > have to merge all the indices. For that I used mergeUtils (which > worksperfect). For now I am using RAMDirectory as the merged indices. > Anyway The indices can grow to a bigger size as time goes. MMapDirectory > again uses memory right? Can It deal with possible out of memory issue? > > I am thinking of using the same Database to store the merged indices. But > the problem is the original sharded indices can be updated, when new > entries come in. So the merged final indices also needs to be updated > accordingly. > > On Thu, Apr 2, 2015 at 4:55 PM, Shai Erera <ser...@gmail.com> wrote: > >> In some cases, MMapDirectory offers even better performance, since the JVM >> doesn't need to manage that RAM when it's doing GC. >> >> Also, using only RAMDirectory is not safe in that if the JVM crashes, your >> index is lost. >> >> On Thu, Apr 2, 2015 at 12:54 PM, Christoph Kaser <lucene_l...@iconparc.de >> > >> wrote: >> >> > Hi Gimantha, >> > >> > why do you use a RAMDirectory? If your merged index fits into RAM >> > completely, a MMapDirectory should offer almost the same performance. >> And >> > if not, it is definitely the better choice. >> > >> > Regards >> > Christoph >> > >> > >> > Am 02.04.2015 um 12:38 schrieb Gimantha Bandara: >> > >> >> Hi All, >> >> >> >> I have successfully setup a merged indices and drilldown and usual >> search >> >> operations work perfect. >> >> But, I have a side question. If I selected RAMDirectory as the >> destination >> >> Indices in merging, probably the jvm can go out of memory if the merged >> >> indices are too big. Is there a way I can handle this issue? >> >> >> >> On Tue, Mar 24, 2015 at 12:18 PM, Gimantha Bandara <giman...@wso2.com> >> >> wrote: >> >> >> >> Hi Christoph, >> >>> >> >>> My mistake. :) It does the exactly what i need. figured it out later.. >> >>> Thanks a lot! >> >>> >> >>> On Tue, Mar 24, 2015 at 3:14 AM, Gimantha Bandara <giman...@wso2.com> >> >>> wrote: >> >>> >> >>> Hi Christoph, >> >>>> >> >>>> I think TaxonomyMergeUtils is to merge a taxonomy directory and an >> index >> >>>> together (Correct me if I am wrong). Can it be used to merge several >> >>>> taxonomyDirectories together and create one taxonomy index? >> >>>> >> >>>> On Mon, Mar 23, 2015 at 9:19 PM, Christoph Kaser < >> >>>> lucene_l...@iconparc.de >> >>>> >> >>>>> wrote: >> >>>>> Hi Gimantha, >> >>>>> >> >>>>> have a look at the class org.apache.lucene.facet. >> >>>>> taxonomy.TaxonomyMergeUtils, >> >>>>> which does exactly what you need. >> >>>>> >> >>>>> Best regards, >> >>>>> Christoph >> >>>>> >> >>>>> Am 23.03.2015 um 15:44 schrieb Gimantha Bandara: >> >>>>> >> >>>>> Hi all, >> >>>>>> >> >>>>>> Can anyone point me how to merge several taxonomy indexes? My >> >>>>>> requirement >> >>>>>> is as follows. I have several taxonomy indexes and normal document >> >>>>>> indexes. I want to merge taxonomy indexes together and other >> document >> >>>>>> indexes together and perform search on them. One part I have >> figured >> >>>>>> out. >> >>>>>> It is easy. To Merge document indexes, all I have to do is create a >> >>>>>> MultiReader and pass it to IndexSearcher. But I am stuck at merging >> >>>>>> the >> >>>>>> taxonomy indexes. Is there a way to merge taxonomy indexes? >> >>>>>> >> >>>>>> >> >>>>>> -- >> >>>>> Dipl.-Inf. Christoph Kaser >> >>>>> >> >>>>> IconParc GmbH >> >>>>> Sophienstrasse 1 >> >>>>> 80333 München >> >>>>> >> >>>>> www.iconparc.de >> >>>>> >> >>>>> Tel +49 -89- 15 90 06 - 21 >> >>>>> Fax +49 -89- 15 90 06 - 49 >> >>>>> >> >>>>> Geschäftsleitung: Dipl.-Ing. Roland Brückner, Dipl.-Inf. Sven >> Angerer. >> >>>>> HRB >> >>>>> 121830, Amtsgericht München >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> --------------------------------------------------------------------- >> >>>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> >>>>> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >>>>> >> >>>>> >> >>>>> -- >> >>>> Gimantha Bandara >> >>>> Software Engineer >> >>>> WSO2. Inc : http://wso2.com >> >>>> Mobile : +94714961919 >> >>>> >> >>>> -- >> >>> Gimantha Bandara >> >>> Software Engineer >> >>> WSO2. Inc : http://wso2.com >> >>> Mobile : +94714961919 >> >>> >> >>> >> >> >> > >> > -- >> > Dipl.-Inf. Christoph Kaser >> > >> > IconParc GmbH >> > Sophienstrasse 1 >> > 80333 München >> > >> > www.iconparc.de >> > >> > Tel +49 -89- 15 90 06 - 21 >> > Fax +49 -89- 15 90 06 - 49 >> > >> > Geschäftsleitung: Dipl.-Ing. Roland Brückner, Dipl.-Inf. Sven Angerer. >> HRB >> > 121830, Amtsgericht München >> > >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> > For additional commands, e-mail: java-user-h...@lucene.apache.org >> > >> > >> > > > > -- > Gimantha Bandara > Software Engineer > WSO2. Inc : http://wso2.com > Mobile : +94714961919 > -- Gimantha Bandara Software Engineer WSO2. Inc : http://wso2.com Mobile : +94714961919
