Hello Dan, I would definitely echo the fuzzing suggestion from Jake.
In addition, in your quest to find broken images, I would suggest grabbing the images from the dvyukov/go-fuzz corpus, which has a bunch of images that are already broken in "creative" ways for you based on the coverage-guided fuzzing that has happened these in the past. For example, here are a bunch of images from there: https://github.com/dvyukov/go-fuzz-corpus/tree/master/jpeg/corpus https://github.com/dvyukov/go-fuzz-corpus/tree/master/png/corpus https://github.com/dvyukov/go-fuzz-corpus/tree/master/gif/corpus You could just use those as inputs to your tests now, even if you don't want to fuzz anything yet. There are also other public fuzzing corpuses for other languages or image libraries that you can find to run your tests against. That go-fuzz corpus repository also contains sample fuzzing functions for jpeg, png, and gif so that you don't have to start from scratch to fuzz your changes. Also, some of those fuzzing functions from the go-fuzz corpus have also somewhat recently been moved into the Go standard library. For example, acln0 moved the png fuzz function from the go-fuzz corpus into the Go standard library. I mention that in part because as part of your effort to make sure you are not introducing bugs, you could consider bringing over the jpeg and gif fuzzing functions from the go-fuzz corpus into the Go standard library, which would help exercise your code (including it would then be a ~2 line PR to google/oss-fuzz to pick up continuous fuzzing funded by Google for those new functions, which would give you on-going coverage of your changes). Finally, one way to fuzz using the fuzzing functions in the standard library (using image/png as the example here) is as follows: $ go get -u github.com/dvyukov/go-fuzz/... $ go get -u github.com/thepudds/fzgo $ fzgo test -fuzz=. image/png fzgo is a wrapper on top of go-fuzz that provides some additional conveniences, but perhaps more interesting is that fzgo is a simple prototype of how cmd/go could behave under the proposal in #19109: #19109 proposal: make fuzzing a first class citizen, like tests or benchmarks https://github.com/golang/go/issues/19109 Sorry for the long post, but I believe you'll get pretty quick payback on the small amount of time it would take to start fuzzing your changes via some of the starting points listed above. Regards, thepudds On Thursday, February 13, 2020 at 1:07:59 PM UTC-5, Dan Sugalski wrote: > > Also it's probably easiest if folks have these handy to just propose an > addition to the test repository at > https://github.com/drswork/image/tree/master/testdata since sending > busted images via email is fraught with all sorts of exciting peril. > On Thursday, February 13, 2020 at 1:05:50 PM UTC-5 Dan Sugalski wrote: > >> Fuzz testing is absolutely reasonable and something I want to set up once >> I've got the first pass of code done. >> >> On Thursday, February 13, 2020 at 11:42:16 AM UTC-5 jake...@gmail.com >> wrote: >> >>> This may be slightly tangential, but this seems like the kind of code >>> that would benefit greatly from fuzz testing, like go-fuzz >>> <https://github.com/dvyukov/go-fuzz>. For this kind of code, go-fuzz >>> <https://github.com/dvyukov/go-fuzz> can really help harden it against >>> bad, malformed and malicious input. I have used it to really good effect. >>> It requires thoughtful choices for the initial corpus, and a machine where >>> you can let it run for a long time, preferable just leave it running for >>> weeks. >>> >>> On Wednesday, February 12, 2020 at 9:52:51 PM UTC-5, Dan Sugalski wrote: >>>> >>>> Specifically ones that are compatible with the Go license. The more >>>> broken they are the better. Bonus points for ones that trigger degenerate >>>> behaviour in the decoding libraries. >>>> >>>> The tldr here is that I'm making a bunch of changes to the basic image >>>> libraries to add in metadata support. Since I'm in there anyway I figure I >>>> may as well add in some safety measures against files that consume an >>>> unreasonable amount of time and/or space to decode. (Or encode if anyone's >>>> got an image.Image that triggers bad behaviour) Having a good suite of >>>> known-bad files to check against will be handy to make sure the code >>>> actually works which is, y'know, kinda nice. >>>> >>>> (I am aware of some bad files kicking around the web, and I've grabbed >>>> some for testing, but none I've run across so far have licenses on them >>>> that'd allow me to toss them up on github as part of the tests for this >>>> stuff. Hence the asking around) >>>> >>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/7ae316a7-14d6-41a2-a1b3-a2b91835bb3b%40googlegroups.com.
