Update: setting the DisabledImages property on the TAdvGlowButton controls to the same thing as the ActionList works. Obviously, it shows the glyphs in colour, not greyed out as I wanted, but at least they don't have that black box around them.
I would still really like to know how to fix this...