After testing my spring connector, it was an improvement over the original connect, but still had some disadvantages.
I redesigned it now and achieved a great improvement:
It now prints easily laying flat. This means the layers of the connector now run at an almost perpendicular angle to the castle pieces (which are usually printed upright), so it's far easier to insert and remove the connectors. You need no brim anymore, which also helps insertion and removal, and saves you the effort of finishing them. The spring, which has to follow the layers, now has more give without danger of breaking. The connector is smaller and only contacts the surfaces defining the connection, so it doesn't rub against any other surfaces.
I'm quite happy with the results, it's really child's play to connect the pieces now ;-)