Re: Is there a way to limit the size of an index?

I'm storing and reading the documents using Compass, not Lucene directly. I didn't touch those parameters, so I guess the default values are being used (I do see cfs files in the index). How the ramBufferSizeMB parameter affect the files size? What value should I use in order to have 6MB files?

Re: Is there a way to limit the size of an index?

Oh, as an excercise I tried to create 6MB files. Using the rule mentioned before, I set the maxMergeMB to 0.6 (and then 0.62, 06.64... 1.8) and used the default mergeFactor - I thought that should do for 6MB files... Michael McCandless-2 wrote: > > What settings did you use (mergeFactor, maxMe

Re: Is there a way to limit the size of an index?

Well, this rule seems not working... I tried to create an index of 90k documents, with different merge factors. Somehow, the files size in the final index were 1MB, or 8MB - nothing in the middle. Am I missing something? Is the best way to really control the files size is to implement a custom Di

Re: Is there a way to limit the size of an index?

Can you explain how the combination of merge factor and max merge size control the size of files? For example, if one would like to limit the files size to 3,4 or 7MB - how these parameters values can be predicted? Michael McCandless-2 wrote: > > > This tells the IndexWriter NOT to merge any

Re: Is there a way to limit the size of an index?

gt; IndexOutput write to the separate 10 MB sized files, then make an > IndexInput that "pretends" they are one file. > > Mike > > On Thu, Jan 7, 2010 at 6:24 AM, Dvora wrote: >> >> Thanks for the reply. >> >> Can you please add some detailed expla

Re: Is there a way to limit the size of an index?

Thanks for the reply. Can you please add some detailed explanations? I'm trying to upload Lucene index to Google App Engine, and the files size must not exceed 10MB. Michael McCandless-2 wrote: > > I don't think this is implemented [yet] today. You'd have to > implement the Directory, IndexI

Is there a way to limit the size of an index?

Hi all, According to the FAQ, "An even more complex and optimal solution: Write a version of FSDirectory that, when a file exceeds 2GB, creates a subdirectory and represents the file as a series of files." Is that solution implemented already? If not, can you guide me please how can I achieve t

Re: How to avoid huge index files

have not used it). > > Also, I have successfully run Lucene on GAE using GaeVFS > (http://code.google.com/p/gaevfs/) to store the index in the Datastore. > (I developed a Lucene Directory implementation on top of GaeVFS that's > available at http://sf.net/contrail). > &

RE: How to avoid huge index files

file (the > file is splitted up into parts of 2 Gigabytes). You could use similar code > here and just use another file, if somebody seeks or writes above the 10 > MiB > limit. Just "virtualize" the files. > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-

Re: How to avoid huge index files

to make a custom Directory impl > that simply splits up files above a certain size. That'd be more > generic and more reliable... > > Mike > > On Thu, Sep 10, 2009 at 5:26 AM, Dvora wrote: >> >> Hi, >> >> Thanks a lot for that, will peforms the exp

Re: How to avoid huge index files

ler to be sure things stay > small enough. > > Note that with this approach, if your index is large enough, you'll > wind up with many segments and search performance will suffer when > compared to an index that doesn't have this max 10.0 MB file size > restriction. > &g

Re: How to avoid huge index files

Hello again, Can someone please comment on that, whether what I'm looking is possible or not? Dvora wrote: > > Hello, > > I'm using Lucene2.4. I'm developing a web application that using Lucene > (via compass) to do the searches. > I'm intending to deplo

How to avoid huge index files

Hello, I'm using Lucene2.4. I'm developing a web application that using Lucene (via compass) to do the searches. I'm intending to deploy the application in Google App Engine (http://code.google.com/appengine/), which limits files length to be smaller than 10MB. I've read about the various policie