https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113156
--- Comment #9 from Andrew Pinski <pinskia at gcc dot gnu.org> --- Maybe the case is mdouble=64 is not being set correctly after the __optimize__ attribute. If I get some more time, I will look into what other targets do for these kind of options.
