Magnet cups are glued in, as they have features that will grip the glue better.
The cups also give the magnet a much stronger field.
The cups also allow you to flip the magnets if you get them reversed, so a little bonus feature.
Magnets themselves are very smooth. and likely to release from the glue, though they could be abraded to help.
The visor takes the magnets directly, I have had the glue fail, but not so much its impractical.