> I had an idea of what one could do, but really, we first need a
> representative test set of image pairs, both pairs with important
> differences and spurious differences, so we can see what a new
> algorithm looks like.
See my other e-mail to Luca how to one of Harm's MR to do that.
On the other hand, it shouldn't be too hard to create a bunch of
LilyPond input files that suppress various grobs (note heads, slurs,
accidentals, etc.), which can be used to create such images – simply
create a fake MR to get this tested!
> The idea is that we'd want to trigger less on diffuse lines of
> difference (shifted staffline), but more on concentrated blobs
> (disappearing symbol). Here is a suggestion
>
> * generate images without anti-aliasing
> * generate the image diff
> * for each changed pixel, count the number of changed neighbor pixels.
> * for each pixel, take max(changed - THRESHOLD, 0)
> * then do MAE or some other metric on that image.
>
> By setting THRESHOLD = 2, you could make single line differences
> disappear. (We'd need to make sure to make the resolution such that
> stafflinethickness is at least 2 pixels, or otherwise dramatic
> changes in staff positioning would not trigger anything.)
Sounds good!
> Alternatively, on the diff image, you could segment the diff into
> connected components, and weight the errors by the minimum of
> {x-size, y-size}. Thus, a line has a diameter of only 1 pixel, but
> missing notehead is much larger.
Sounds good, too. However, my knowledge of such algorithms is far too
limited to even have an opinion.
Werner
