On Thursday, May 12, 2016 12:03:21 PM CEST johannes hanika wrote: > heya, > > sorry for the delay, i had half of this mail sitting in my drafts > folder for quite some time now.
Hi, its all right. This week's tutorial I have to give to fulfill my teaching duty took quite some time out of my pocket. Thus, I have not progressed any further with the LLF. But I have high hopes for the coming week. :) Thanks for your comments. I come back to you, once there is something to discuss. Cheers, Wolfgang P.s.: I forked dt on github. I let you know once there is a development branch for the LLF. > > On Tue, May 10, 2016 at 12:54 AM, Wolfgang Mader > > <[email protected]> wrote: > > Hi Johannes, > > > > I write this mail personally to you since I am not sure if my excerpt of > > the paper is too detailed for the list. As you send me the paper, I > > assumed you the expert on the topic. > > heh. i did see the talk five years ago, and thought i would understand > it back then. let's see how much detail i remember :) > > > I hope this is fine for you. If you think the list > > would benefit from this mail, feel free to forward it or let me know, such > > that I can post it. > > > > I have summarized my current understanding of the algorithm and pointed > > out > > points which are unclear to me. I hope this gives us a nice basis for > > discussion w/o assuming something the other might not know/be aware of. > > > > > > 1) Obtain the Gaussian pyramid > > ------------------------------ > > Starting from the original image I with i rows and j columns, the n-th > > level of the Gaussian pyramid is a downsampled version of I with i/(2**n) > > rows and j/(2**n) columns, yielding ij/(4**n) pixels at n-th level; x**y > > denotes x raised to the power of y. In order to avoid aliasing, the image > > must be downsampled such that the Nyquist-frequency is halved from level > > n to n+1. The filter doing this must not introduce spurious structures at > > a coarser scale.[1] > yes. > > > To this end, the scale space community (if there such a thing) has mainly > > settled on Gaussian filters, which has been claimed the unique solution > > for > > this problem. This claim, however, seems disputable.[1] > > right. i don't think it matters much in practice as long as you're > blurring (and not introducing frequency by some ringing filter). > > > In the reference code, downsampling is performed by a 2D separable low > > pass > > filter constructed as the outher product of the parameter vectors of a 1D > > 5-tab low pass filter. I guess, separable meens that the covariance > > matrix of the 2D Gaussian function is has zero off-diagonal entries. Is > > this correct? > separable means you can first do a 1D blur in x and another 1D blur in > y direction. that is, you separate the 5x5 2D filter into two 5-wide > 1D filters. which is of course a little bit faster. i think you can > show that a gaussian is pretty much the only thing that separates from > a radially symmetric filter kernel to two 1D gaussians, without > introducing some square-shaped artifacts. > > > This is the first point where we can play with parameters. However, > > whatever we implement here, we must respect the Nyquist frequency. > > it's really kind of hard not to, no? i mean, you have a signal with 2 > pixels, so the maximum frequency you get out of this is just under > wavelength=2. now you average those two and get a signal with max > wavelength=1 ..? > > > 2) Obtaining the Laplacian pyramid > > ---------------------------------- > > The n-th level of the Laplacian pyramid is defined as > > > > L(n) = G(n) - upsample(G(n+1)) > > > > where upsample() is an operator that doubles the image size in each > > dimension. Thus G(n) and upsample(G(n+1)) have the same number of pixels. > > The top-most level L(N) equals G(N), while L(0) = G(0) - upsample(G(1)). > > Starting with G(N), the Laplacian pyramid can be collapsed > > > > G(n) = L(n) + upsample(G(n+1)) > > > > such that finally > > > > G(0) = L(0) + upsample(G(1)) > > > > the original image is resurrected. > > yes, multiresolution, much like wavelets would work, too. > > > 3) Detecting edges > > ------------------ > > Edges are detected in a color space (rgb, luminosity, etc.) appropriate > > for > > the problem at hand. As darktable already offers many different color > > spaces, this should nicely fit darktable's design. It is assumed, that > > fine detail generates smaller discontinuities in this color space than > > edges do. As we probably want to keep detail, this is the second point of > > interaction we can play with. > > yeah, i would try to do it in our Lab pipeline, and detect edges > mainly via L. not sure whether we want the pyramid to work on 3d > colour at all, or just on the lightness. > > > 4) Altering edges > > ----------------- > > The amplitude of detected edges shall be adjusted while the shape of the > > edge (position, shape, direction of its gradient) shall remain > > unmodified. Edges with different profiles end up in different levels > > L(n). Therefore, altering the coefficients in the Laplacian pyramid > > directly while preserving the properties of the edge is tricky. > > yes. > > > An easier approach is to alter the original image in its color space is > > such a way that it locally equals our target. This can be compression or > > enhancement or whatever else comes to mind. New Laplacian coefficients > > from this altered image are then determined and replace the original ones > > in the Laplacian pyramid. This operation is local as further discussed in > > 6). > > yes, the tricky part is to make the laplacians local. > > > 5) Preserve detail > > ------------------ > > Detail is lost by the compression of the original image. > > by compression you mean reduction of overall contrast by compressing > dynamic range with, for instance, a tonecurve on L. > > > The paper mentions a > > nifty trick to avoid this (Paragraph: Detail preservation). However, for > > the current discussion, this is not that important. I skip it for now and > > have a closer look at it once the rest of the algorithm is clear to me. > > okay. > > > 6) Local Laplacian filtering > > ---------------------------- > > Each pixel (k,l,n) of the n-th level of the Laplacian pyramid is obtained > > from an intermediate image I' which is explicitly generated to obtain > > this single pixel. Here, the paper is a bit blurry, such that I have to > > consult the reference code in order to figure out what really is going > > on. If you can fill me in here, I would appreciate. However, the > > transformation applied to I in order to obtain I' is key to the local > > Laplacian filter. By adjusting the parameters of this local > > transformation, detail, tone or both can be either smoothed or enhanced. > > did you see https://youtu.be/t4mbrKYRmvs ? > > i think it nicely visualises the optimised version of creating the > laplacian pyramids (by in fact only computing the gaussian pyramids > and evaluating the laplacians on the fly). the sampling step in the > end is of course not explained in this video, but i think it's > probably a good idea to follow the paper with the optimised > implementation that is cited below this video (your [2]). > > > 7) Conclusion > > ------------- > > To me, a variate of applications seem possible. Depending on which > > information of the image (color, luminosity, etc.) we include in the > > pyramids, different aspects of the image should be affected. Also, by > > selecting only some of all possible levels of the Laplacian pyramid, we > > can limit the effects to specific frequency bands of the image. This > > should be similar to what we can do in the wavelet-based Equalizer > > module. > > indeed. > > > 8) Plan of action > > ----------------- > > I found another publication about a fast implementation of the > > algorithm.[2] I want to read this and further dig through the code, both > > the Matlab version and what you send me. > > > > Meanwhile, I would be happy to read your comments if you have any. Once I > > am all clear on the algorithm, I for sure have some questions on how to > > manage the entire image and parts of it in the pixel pipeline. It would > > be great if you could help me there. > > absolutely. i would consider that the easy part :) > > cheers, > jo > > > This is all for now from my side. > > Cheers, > > Wolfgang > > > > > > > > > > [1] https://en.wikipedia.org/wiki/Scale_space#Why_a_Gaussian_filter.3F > > [2] http://www.di.ens.fr/~aubry/llf.html > > ___________________________________________________________________________ > darktable developer mailing list > to unsubscribe send a mail to [email protected]
signature.asc
Description: This is a digitally signed message part.
