[
https://issues.apache.org/jira/browse/LUCENE-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318434#comment-14318434
]
Robert Muir commented on LUCENE-6241:
-------------------------------------
Just illustration: remove the isDirectory() call in my patch and run
TestRAMDirectory, the exception would look like this:
{noformat}
java.io.IOException: Is a directory:
NIOFSIndexInput(path="/tmp/lucene.store.TestRAMDirectory
F412F4510E9B7B4-001/testsubdir-001/subdir")
at
__randomizedtesting.SeedInfo.seed([F412F4510E9B7B4:3E7CBDE940612694]:0)
at
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:190)
at
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:160)
at
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:116)
at
org.apache.lucene.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:140)
at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:275)
at org.apache.lucene.store.Directory.copyFrom(Directory.java:156)
at org.apache.lucene.store.RAMDirectory.<init>(RAMDirectory.java:102)
at org.apache.lucene.store.RAMDirectory.<init>(RAMDirectory.java:96)
at
org.apache.lucene.store.TestRAMDirectory.testCopySubdir(TestRAMDirectory.java:78)
{noformat}
Its not something we can easily "upgrade" to a better exception.
> don't filter subdirectories in listAll()
> ----------------------------------------
>
> Key: LUCENE-6241
> URL: https://issues.apache.org/jira/browse/LUCENE-6241
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Attachments: LUCENE-6241.patch, LUCENE-6241.patch
>
>
> The issue is, today this means listAll() is always slow, sometimes MUCH
> slower, because it must do the fstat()-equivalent of each file to check if
> its a directory to filter it out.
> When i benchmarked this on a fast filesystem, doing all these filesystem
> metadata calls only makes listAll() 2.6x slower, but on a non-ssd, slower
> i/o, it can be more than 60x slower.
> Lucene doesn't make subdirectories, so hiding these for abuse cases just
> makes real use cases slower.
> To add insult to injury, most code (e.g. all of lucene except for where
> RAMDir copies from an FSDir) does not actually care if extraneous files are
> directories or not.
> Finally it sucks the name is listAll() when it is doing anything but that.
> I really hate to add a method here to deal with this abusive stuff, but I'd
> rather add isDirectory(String) for the rare code that wants to filter out,
> than just let stuff always be slow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
