Re: Lucene index directory grows and shrinks

2019-11-04 Thread Erick Erickson
Here’s a neat visualization: http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html The short form is this: - A “segment” is all the files with a particular prefix in your index directory, e.g. _12ey1* is one segment - Segments are created as documents are indexed and

R: Lucene index directory grows and shrinks

2019-11-04 Thread Raffaele Gambelli
For what you know, is this behaviour which you defined "typical" described deeply somewhere? It is foundamental for me to better understand it even to know how big an index can grow, in a way that I can allocate the right disk space. Thank you very much -Messaggio originale- Da: Raffae

R: Lucene index directory grows and shrinks

2019-11-04 Thread Raffaele Gambelli
Thanks for your quick reply, I'm quite a beginner in Lucene concepts, Jackrabbit hides almost all about the way it uses Lucene internally. Anyway here it is the size of each sub-directory in my index, please note the bigger one, 25G, is it normal? ...repository/workspaces/default/index$ du -h

Re: Lucene index directory grows and shrinks

2019-11-04 Thread Atri Sharma
This are typical symptoms of an index merge. However, it is hard to predict more without knowing more data. What is your segment size limit? Have you changed the default merge frequency or max segments configuration? Would you have an estimate of ratio of number of segments reaching max limit / to

Lucene index directory grows and shrinks

2019-11-04 Thread Raffaele Gambelli
Hi all, I'm using Jackrabbit 2.18.0 which uses lucene-core 3.6.0. I'm working on an application that has reached 37 G of directory index, a few days ago, disk occupancy has quickly reached 100% and then returned to pre-growth employment. I believe that was caused by a rapid growth of Lucene in