[
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14904338#comment-14904338
]
Dawid Weiss commented on LUCENE-6813:
-------------------------------------
I don't fully understand the problem but to me OfflineSorter is thread safe --
it takes input and output paths, then potentially creates some intermediate
files which should never cause any threading problems because they're created
atomically by the file system. OfflineSorter also makes best effort to delete
these files. If your output already exists, it should be overwritten... where
is the thread safety problem?
As for Windows and the pending delete queue, can we pinpoint when this is
happening (is it a leaked file handle, lock)? Perhaps there is a better fix to
just cater for the delay of file/ folder deletion in Windows (assuming this is
a documented feature)? If Files.delete returns and the file is not deleted,
this seems like a bug in the JDK to me?
http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#delete(java.nio.file.Path)
> OfflineSorter.sort isn't thread-safe
> ------------------------------------
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe. It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
