I got this solenoid from an old laser printer, so I cannot provide details on how to get it. Its aproximate dimensions are 26 x 20 x 30 mm. You can try to glue yours in place, or drill some holes and screw it in place.
In my setup (which is not the most reliable), the solenoid is a pull one, with spring return. When the solenoid is activated, the carriage's spring presses the pen against the paper. When the solenoid is released, solenoid's spring pushes a lever up, with in turn lifts the pen, overcoming the carriage's spring force. It works for me, but could be improved.
If you use 'liquid ink' pens (wich don't need to be pressed much against the paper to write as bic pens do) you can get rid of the carriage spring and use a much weaker solenoid.
Good luck with your build!