I filed an issue (https://github.com/golang/go/issues/24015) that is looking like a bug in RHEL 7 when running getdents64 with too small a buffer.
Brief summary: Go pulls data 4k at a time using the getdents64 system call on Linux. For some reason the response is always a pattern where the first response nearly fills the 4k buffer but drops a file, then the second response gets about 1500 more bytes and has no issues. This pattern repeats until all dir entries have been read (other than those which were dropped). If you update the buffer to 32k, the process works fine, pulling in roughly 5.5k bytes per iteration and not dropping any files. I have no idea why CIFS is returning such an odd number of bytes per iteration. But I'm not sure what to do next. Maintaining our own fork of Go is not really feasible, obviously, but this bug is very edge-case and not Go's fault anyway. That said, it is a major blocker for us unless we switch off of RHEL for multiple projects, which is also infeasible. Is a kernel-bug workaround something that could get into Go? I have two possible solutions in Go core: - Change os/dir_unix.go to allocate blockSize*8 bytes (32k, much like the C readdir function uses) - Use build tags to set a new constant for this specific case, where it's blockSize*8 on amd64 systems and blockSize on all other systems I don't really feel comfortable making the assumption that no other architectures will have this problem, but I also feel like there must be a reason for such a small buffer by default, so some of them must need the extra performance or tiny reduction in memory or something. Would either of these solutions be acceptable as a pull request? At this point I'm liable to reimplement os.File.Readdir, but that's probably going to be a terrible idea long-term. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
