Hi all, Replying to this 9-year-old thread because the original question has finally been answered.
Back in 2015, the consensus was that HDF5 is "so complicated that there is only one implementation" and too difficult for pure Go. I'm happy to report that's no longer true. WHAT EXISTS NOW (2025): A pure Go HDF5 library with full read support and beta write support: Repository: https://github.com/scigolib/hdf5 Read: Feature-complete (superblock v0/2/3, all layouts, compression, attributes) Write: Beta (v0.11.1-beta - chunked datasets, GZIP, dense groups, attributes) Example - Reading: file, _ := hdf5.Open("data.h5") dataset := file.Datasets["/temperature"] data := dataset.Data() // []float64, []int32, etc. Example - Writing (beta, but functional): file, _ := hdf5.CreateForWrite("output.h5", hdf5.Truncate) file.CreateDataset("data", myData, hdf5.WithChunked([]uint64{100, 100}), hdf5.WithCompression(6), ) HOW IT WAS DONE: The C library (https://github.com/HDFGroup/hdf5) served as reference implementation. Instead of "figuring out" the format from scratch, we ported proven algorithms to Go. Format spec + reference code = solvable problem. Development time: ~1 year from concept to write MVP (with AI assistance for rapid prototyping). WHY IT MATTERS: - No CGo = actual cross-compilation, no C dependencies - Type safety = Go's compiler catches HDF5 format errors at compile time - Standard library integration (io.ReaderAt, encoding/binary) - Single binary deployment CURRENT STATUS: - Test coverage: 70-88% depending on package - Platforms: Linux, macOS, Windows - Recognition: HDF Group acknowledged it on their forum ( https://forum.hdfgroup.org/t/loking-for-an-hdf5-version-compatible-with-go1-9-2/10021/7 ) - Beta limitations: Some write features in progress (dense storage read-modify-write, h5dump compatibility) FOR THE SCIENTIFIC GO COMMUNITY: If you're working with HDF5 files and want to avoid CGo, this is now viable. Looking for beta testers with real-world datasets (astronomy, climate, genomics, etc.). Installation: go get github.com/scigolib/[email protected] The format is indeed complex, but it's been tackled. Thought this group might appreciate the update after 9 years. Best, [Your name] P.S. NetCDF4 being "stripped down HDF5" (as mentioned in the original thread) means this library could potentially support it too, though that's not implemented yet. On Tuesday, 28 December 2010 at 03:47:30 UTC+3 darenw wrote: > How well would a Go program be able to read and write the major > scientific file formats? Do libraries already exist? > > Although I've read that Go might not be the best choice for massive > number crunching, many science and engineering apps tody have complex > GUIs, connect to other apps via dbus or other interconnection systems, > and do a lot of threading for example to let the user continue playing > with the GUI while data is saved to a file in the background. The > thing I'm working on currently has over 600 classes in a messy > hierarchy, many objects having pointers to other objects to access > methods that only return pointers to yet other objects, and most of > the real work done is done by short snips of code, one to several > lines, scattered hither thither and yon throughout a very large > directory tree. > > I'm fantasizing the whole thing rewritten in Go might be 1/4 the size, > 1/10th the bugs, and way easier to maintain. Never mind performance > of huge arrays of numbers - that can be dealt with in various ways. > I might try a small demo program in Go, but it'll be far easier if I > can at least read FITS files. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/5a80a4e1-9b08-44dd-bd47-936977fa97aen%40googlegroups.com.
