> (a) Accessing index files over NFS from a "single" physical process on a
> single computer is safe and can be made to work.
To add: This means writing only. Reading is fine from as many threads as you
like - and using MMapDirectory for best performance. The problem with writing
is that lockin
OK, so it sounds like I'm hearing that
(a) Accessing index files over NFS from a "single" physical process on
a single computer is safe and can be made to work.
(b) Accessing index files over NFS from "multiple" processes/machines might
be problematic
(c) In all cases, the performance would be l
John,
Are you indicating that later Lucene releases might have a config setting
that can control the write I/O timeout? If so, do you happen to know where
it is or how to set it? I did quick Googling, but all I get back is the
write lock timeout which is set to one second by default.
Thanks
/Jong
We've been in production on Lucene over NFS for about 4 years now. Though
we've had performance issues related to NFS (similar to those mentioned on
this thread), we've only seen some reliability issues. Index writing I/O
timeout exceptions are the primary issue. We've addressed these by
impleme
Ok, that saves you from concurrency issue, but in my experience is just
much slower than local file system, so still NFS can be used but with some
tradeoff on performance.
My 2 cents,
Tommaso
2012/10/2 Jong Kim
> The setup is I have a home-grown server process that has exclusive access
> to the
Uwe,
Thanks for the detailed information. Are you aware of an existing
implementation of the IndexDeletionPolicy interface that is "known" to work
reliably with NFS?
/Jong
On Tue, Oct 2, 2012 at 9:01 AM, Uwe Schindler wrote:
> There are no real issues with NFS regarding safety of the data. The
>
The setup is I have a home-grown server process that has exclusive access
to the index files. All reads and writes are done through this server. No
other process is reading the same index files whether it's local or over
NFS.
/Jong
On Tue, Oct 2, 2012 at 8:56 AM, Ian Lea wrote:
> I agree that rel
My Lucene index is accessed by multiple threads in a single process.
/Jong
On Tue, Oct 2, 2012 at 8:45 AM, Paul Libbrecht wrote:
> I doubt NFS is an unreliable file-system.
> Lucene uses normal random access to files and this has no reason to be
> unreliable unless bad things such as network dr
There are no real issues with NFS regarding safety of the data. The problem
with NFS is the following (maybe it is fixed in NFS4, I have no idea):
Lucene deletes index files while they are in use, which is perfectly fine for
local file systems (because the inode is still alive, although it is no
I agree that reliability/corruption is not an issue.
I would also put it that performance is likely to suffer, but that's
not certain. A fast disk mounted over NFS can be quicker than a slow
local disk. And how much do you care about performance? Maybe it
would be fast enough over NFS to make t
I doubt NFS is an unreliable file-system.
Lucene uses normal random access to files and this has no reason to be
unreliable unless bad things such as network drops happen (in which case you'd
get direct failures or timeouts rather than corruption). I've seen fairly
large infrastructures being b
Thank you all for reply.
So it soudns like it is a known fact that the performance would suffer
rather significantly when the index files are accessed over NFS. But how
about reliability and robustness (which seems even more important)? Isn't
there any increased possibility for intermittent errors
My experience in the Lucene 1.x times were a factor of at least four in writing
to NFS and about two when reading from there. I'd discourage this as much as
possible!
(rsync is way more your friend for transporting and replication à la solr
should also be considered)
paul
Le 2 oct. 2012 à 11
You'll certainly need to factor in the performance of NFS versus local disks.
My experience is that smallish low activity indexes work just fine on
NFS, but large high activity indexes are not so good, particularly if
you have a lot of modifications to the index.
You may want to install a custom
14 matches
Mail list logo
