A few years ago, I uploaded a parallel “find duplicate files in a filesystem” utility to GitHub. It is called “dup” for “find duplicates.”
https://github.com/MichaelTJones/dup/blob/master/dup.go In that program, which first groups files by size, then compares the first few bytes, then compares hashes, and then entire files byte-by-byte (all options), is this code: if bytes.Compare(a, b) != 0 { log.Printf("Congratulations! You have found two files that differ(*), yet have\n") log.Printf("the same hash value. Publish these two files for a moment of fame:\n") log.Printf(" file %q\n", path[0]) log.Printf(" file %q\n", path[i]) log.Printf("* Or, the files were modified while this program was executing, in\n") log.Printf(" which case there is no fame to be had. Just run the program again.") return false } It reflects common understanding about the probability of hash collisions when using good hashes. -- Michael T. Jones From: <golang-nuts@googlegroups.com> on behalf of "atd...@gmail.com" <atd...@gmail.com> Date: Wednesday, August 3, 2016 at 4:57 PM To: golang-nuts <golang-nuts@googlegroups.com> Subject: [go-nuts] Re: How to check that a Go package has not been modified? Yes, I suppose that clamping the file size, and the requirement that the package must pass compilation, even md5 shall do. Thanks to the avalanche effect. On Wednesday, August 3, 2016 at 9:36:34 PM UTC+2, Hotei wrote: The fact that collisions are possible does not make them "easy to create" especially when you add the compileable requirement. If you're uneasy about md5 you could always use more bits - like SHA1 used by "git" or SHA256 (or larger) if you're really paranoid. On Wednesday, August 3, 2016 at 1:14:53 PM UTC-4, atd...@gmail.com wrote: Would a md5 hash suffice? I mean, it is probably easy to create collisions, but the source still needs to compile so... Or an md5 hash and using go/types to make sure that any object escaping the package boundaries is still present and none other have been added. Any idea? -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
