Thanks Robert.
I think these valuable comments need to be placed on javadocs for future
references.
i think i am getting enough info for making a decision:
i will use MMapDirectory without setPreload and i hope my index will fit
into the RAM.
i plan to post a blog for findings.
Best regar
On Mon, Dec 14, 2020 at 1:59 PM Uwe Schindler wrote:
>
> Hi,
>
> as writer of the original bog post, here my comments:
>
> Yes, MMapDirectory.setPreload() is the feature mentioned in my blog post is
> to load everything into memory - but that does not guarantee anything!
> Still, I would not recom
I see, i think i will use first way the constructor woith MMap and i
will not use setPreload api to avoid slowdowns.
yes, i was expecting a warning from eclipse in the second usage but
nothing came up.
Thanks for the clarifications.
Best regards
On 12/14/20 2:55 PM, Uwe Schindler wrote:
H
Hi,
> Thanks Uwe, i am not insisting on to load everything into memory
>
> but loading into memory might speed up and i would like to see how much
> speedup.
>
>
> but i have one more question and that is still not clear to me:
>
> "it is much better to open index, with MMAP directory"
>
>
This also brings me another question:
does using MMap over FSDirectory bring any advantage with or without tmpfs?
Best regards
On 12/14/20 2:17 PM, Jigar Shah wrote:
Thanks, Uwe
Yes, recommended, tmpfs/ramfs worked like a charm in our use-case with a
read-only index, giving us very high-thro
Thanks Uwe, i am not insisting on to load everything into memory
but loading into memory might speed up and i would like to see how much
speedup.
but i have one more question and that is still not clear to me:
"it is much better to open index, with MMAP directory"
does this mean i should n
Thanks, Uwe
Yes, recommended, tmpfs/ramfs worked like a charm in our use-case with a
read-only index, giving us very high-throughput and consistent response
time on queries.
We had to have some redundancy to be built around that service to be
high-available, so we can do a rolling update on the r
Hi,
as writer of the original bog post, here my comments:
Yes, MMapDirectory.setPreload() is the feature mentioned in my blog post is
to load everything into memory - but that does not guarantee anything!
Still, I would not recommend to use that function, because all it does is to
just touch ever
Hi,-
are there some examples on how to use MMapDirectory during indexing (i
used the constructor to create it) and search?
what are the best practices?
should i repeat during search what i did during indexing for
MMapDirectory i.e, use the constructor to create the MMapDirectory
object by
Thanks Jigar, these are great notes, observations, experiments to know
about and they are very very valuable,
i also plan to write a blog on this topic to help Lucene advance.
Best regards
On 12/14/20 12:44 PM, Jigar Shah wrote:
I used one of the Linux feature (ramfs, basically mounting ram
I used one of the Linux feature (ramfs, basically mounting ram on a
partition) to guarantee that it's always in ram (No accidental paging ;)
cost too).
https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux
WARN: Only use if it's a read-only index and can fit in ram and have a
back-up c
Thanks Mike, appreciate the reply and the suggestions very much.
And Your article link to concurrent search is amazing.
Together with in memory and concurrent index (especially in read only mode)
these will speed up Lucene queries very much.
Happy Holidays
Best regards
On 12/14/20 10:12 AM,
Hello,
Yes, that is exactly what MMapDirectory.setPreload is trying to do, but not
promises (it is best effort). I think it asks the OS to touch all pages in
the mapped region so they are cached in RAM, if you have enough RAM.
Make your JVM heap as low as possible to let the OS have more RAM to
13 matches
Mail list logo
