Sam, I do not think that I can use LibRAW (or LibTIFF) on appengine. And I aim to write and run some apps on appengine that will let me do some things with my RAW images (not just the thumbnail or the jpeg previews).
Extracting the data (and metadata) is the easy part. I have written things in Go that can do that all day long (makernotes are usually always more involved of course). Knowing what to do with the data after you extracted it is the piece that makes this more complex. Most of the libraries (LibTIFF and LibRAW) out there tend to enumerate all of the things they want to support and how to handle them. I want something much more flexible than that. I am fairly familiar with the different variations of metadata and embedded structures from different camera manufacturers and how they like to abuse some formats. Not just that, but as you say, most of them are really just TIFFs under the hood. For the files that are not TIFF, this becomes much easier as long as you have the file format specification. My initial aim is to solve the TIFF problem in a flexible way that all TIFF based use cases can benefit from, not just camera raw files. I have seen people write their own TIFF implementations just to handle a single case (e.g. EXIF parsing, image preview extraction, bio-medical imaging, GIS imaging). None of these provide a common foundation that all cases can use. This is not an easy problem to solve, but I think it can be made more elegant than existing implementations. I see TIFF as a multi-layered issue. The first and simplest piece is the basic file structure. The structure of the tiff header and top level IFDs are pretty straightforward (even in BigTIFF) and easy to parse. This should be its own package. Why? Because EXIF/GPS will use it and others may also. A lot of people do not realize it, but almost every camera phone (and consumer and professional camera) that produces a JPEG, also embeds a TIFF file inside the APP1 marker segment of that JPEG. But instead of containing what people think of as a traditional TIFF image, it contains EXIF metadata and sometimes GPS metadata. The basic structure is the same, the data being represented might be different. The second piece is the types, tag IDs, tag spaces, tag sets, and tag values. This is really an adaption layer on top of the file structure that leads to metadata interpretation and/or image data with values to process and apply to the image content found within. This could be a separate package, but could also live with the package containing the file structures. The field types are especially beneficial to interpreting fields instead of just entries in an IFD (they are different). The third piece is the image handling and processing. This can definitely be its own separate package. The idea is that in the second piece, you interpreted all of the tags to mean something, but the image layer knows what to do with that data. The basic package should only handle baseline and extended tiff tags and values. And this is where I envision the specifics begin to break out. The idea I have would involve being able to register new types and new tags with the base tiff package. The baseline and extended tags (and their values) would be baked right into the library, but there will be room for growth and functions to register new tag IDs, values, and types. Add to this the registration of handlers to take over when certain tags and values are seen. Some examples: 1. Tag 50706 is DNGVersion. If seen, pass a TIFF object over to a DNG image handling library for processing. 2. Tag 271 & 272 are Make & Model respectively. If the Make is "NIKON CORPORATION" and the Model is "NIKON D800", then pass the TIFF object over to a handler registered to process files from a Nikon D800. The idea here being that, if we do not support some new tag value or camera in the base package, you can write your own package to implement the pieces you need and register it with the base package while all of the other plumbing just works. There will be a lot of overlap in functionality in those libraries at first. But much of that can be centralized where it makes sense and be made accessible for future common use (e.g. compression values referencing compression methods). I know it is a lot of work. I have been thinking about this for a few years and even started prototyping some of it, but I really would like to see it happen. I have hit a few roadblocks in design and implementation, but more importantly I simply do not have the time to tackle this myself anymore. I also agree with Peter Herth that good quality native libraries are good for the existing community as well as for enticing newcomers to the language. Thoughts from anyone on this? On Tue, Jul 26, 2016 at 11:01 PM, Sam Whited <s...@samwhited.com> wrote: > On Tue, Jul 26, 2016 at 9:35 PM, Jonathan Pittman > <jonathan.mark.pitt...@gmail.com> wrote: > > Figuring out how to handle this problem for one specific camera's raw > files > > is not too difficult. Figuring out how to do this to handle the > majority of > > cases requires a bit more work. > > I know libraw has its issues (and lots of them), and I completely > understand wanting a native Go way to get raw data, but I highly > recommend that you don't. There's absolutely no standard for raw image > file formats (most of them are slightly-off-standard variants of TIFF > with various silly transformations applied to the metadata for no > reason) between camera vendors, or even between different models made > by specific cameras. You'll be wasting a lot of time and effort trying > to duplicate the work that libraw has already done for you (and done > well). Instead, consider contributing to libraw if you see cameras > that it doesn't support. > > That being said, if you wanted to talk about processing raw bayer data > without using cgo, I'd say go for it! Use libraw to actually extract > the data, then do the image manipulation itself in Go instead of using > libraws dcraw emulation layer. > > I'm one of the authors of RawKit > (https://rawkit.readthedocs.io/en/latest/), a Python library for photo > manipulation that ships with libraw ctype bindings, and this is the > approach we will eventually take (although we'll most likely be doing > the photo manipulation in Rust, the raw extraction with libraw, and > just writing Python bindings for both). > > —Sam > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
