[
https://issues.apache.org/jira/browse/LUCENE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13959865#comment-13959865
]
Michael McCandless commented on LUCENE-5561:
--------------------------------------------
Thanks for the feedback Rob!
bq. Can this be a "positive test" (e.g. LINUX | SOLARIS | MACOX) rather than
have tests failing on any unknown platform? Its better to just be safe...
Will do.
bq. So I have a hard time understanding why you don't do this in dump().
The problem is (I think?) I have to alloc a separate byte[] in addition to the
ByteBuffer (I can't pass a ByteBuffer to CRC32.update), and then dump first
dumps to that byte[], crc.update that, and then writes that to the file. It
sounds like this (double-buffering on write) is the lesser evil and so I'll
just do that ...
bq. This will not work except on your computer.
Woops :) I'll base it from build.dir.
bq. Also, any existing stuff in LD_LIBRARY_PATH should be preserved, so you
have to use some 'path' stuff in ant to do it correctly.
OK I'll figure out how to pass through the incoming LD_LIBRARY_PATH.
bq. Long term for the future, if dirs like this need a fixed-size buffer, it
would be cool to have a BufferedOutput that works a little differently in that
way. I am not really sure, but i think the HDFSDirectory stuff has code
duplication for this same reason. You'd have to handle the "end" special though
I guess...
+1
I actually had an ALMOST working version of "just subclass
BufferedIndexOutput", and it used a sneaky trick to handle that last buffer
flush on close, but BIO.writeBytes is not aligned today. I agree it'd be nice
to have an "always aligned" BufferedIO.
bq. Also long term it would be nice if the JNI could be removed, and it was
possible to do just some evil reflected call to open with O_DIRECT and get
pageSize and so on. This would actually be safer, too.
+1, I put a TODO about this, it looks very possible, based on the
sun.nio.fs.UnixNativeDispatcher, though I'm not sure how portable that is to
other JVMs...
> NativeUnixDirectory is broken
> -----------------------------
>
> Key: LUCENE-5561
> URL: https://issues.apache.org/jira/browse/LUCENE-5561
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: 4.8, 5.0
>
> Attachments: LUCENE-5561.patch, LUCENE-5561.patch
>
>
> Several things:
> * It assumed ByteBuffer.allocateDirect would be page-aligned, but
> that's no longer true in Java 1.7
> * It failed to throw FNFE if a file didn't exist (throw IOExc
> instead)
> * It didn't have a default ctor taking File (so it was hard to run
> all tests against it)
> * It didn't have a test case
> * Some Javadocs problems
> * I cutover to FilterDirectory
> I tried to cutover to BufferedIndexOutput since this is essentially
> all that NativeUnixIO is doing ... but it's not simple because BIO
> sometimes flushes non-full (non-aligned) buffers even before the end
> of the file (its writeBytes method).
> I also factored out a BaseDirectoryTestCase, and tried to fold in
> "generic" Directory tests, and added/cutover explicit tests for the
> core directory impls.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
