[
https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905093#comment-14905093
]
Dawid Weiss commented on LUCENE-6813:
-------------------------------------
bq. I think the problem is (maybe) that OfflineSorter.sort currently removes
its output path well before writing to it, and so if the caller is relying on
Files.createTempFile to "pick" a unique filename across threads, which BKD is
doing, then this can illegally re-use the same output Path across threads.
Ok, I think I understand you now. In that case indeed OfflineSorter.sort
shouldn't be removing the output path and calling Files.move with
REPLACE_EXISTING. I don't think an atomic move is required (since we don't care
about other processes observing partially moved/copied file).
> OfflineSorter.sort isn't thread-safe
> ------------------------------------
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe. It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
