Anyway, attached is another version of the patch. The only drawback is that toplevel selection (when cur.inset() == this) makes background disappear.
I can not continue. It is amazing how many times this InsetMathHull::draw function is called for a simple selection. If you select within InsetMathNested, InsetMathNested::drawSelection is always called after InsetMathHull::draw so selection would appear (select within a matrix, e.g.). However, on the top level math inset, InsetMathChar::drawSelection is empty so selection is NOT draw within InsetMathHull::draw and the selection is drawn elsewhere.... Adding a InsetMathChar::drawSelection should help but the bv.inset() is always zero ... If mathbg is easy, it should have been implemented long ago... Bo