You can remove them, but its really not a good idea
Crusader said: ↑
And as for the internals, if you check the controler teardown posterd on this forum it looks like it can take some abuse.
I, too, have taken both the Mavic and the controller apart .
After I saw the internals, I decided to stop removing the sticks. Pushing the sticks down requires force, but there's no way to give support for the socket or the connected parts at all. Your only way to have leverage is to push back with the whole controller, essentially forcing the external components inside the controller. Doing it a few times is okay, the controller is well built, but doing it every time you pack it isn't a good idea. At least I wouldn't recommend it.