Charles, I was debugging something that had the same properties.
I was following the comm struct for corruption-on-free in the read phase of a write-then-read test, in mpich-1.2.7p1 (and mpich-1.2.6). It only manifested on 64bit nodes, and only when compiled on 64bit nodes. The binary when compiled on 32bit and run on 64bit did not fail. My smallest test case was two machines, and failure was intermittant, both in reported stack trace, and upon size of the buffers being used. The app that triggered this was a benchmark from one of the (US) labs used for benchmarking i/o on a storage product. Eric