I am agree with this for moderately optimizing compilers. But for highly optimizing compilers it might be no true. Intel generates much better and bigger code than gcc. Although it might be mostly because of code versioning (including one for different subtargets).
I don't think this is true if you select the appropriate option in ICC to generate code for just one target, but of course if you let ICC generate code for multiple targets (e.g. GenuineIntel with SSE vs AuthenticAMD without SSE), then of course you get larger objects, since you have a run time test and then essentially two separate compilations of the same code in the same object.