I made some adjustments:
I reduced the hole size for the idlers to a M3 hexagon screw. It fits OK and taps the plastic on the other side.
I increased the clearance for the idlers a bit. Roughly.
I inserted a 4 mm spacing between the two parts and made the motor's screws adjustable. This way the two X gantry parts may have a small gap between them (or maybe not) but the motor holds the whole assembly together anyway, so it allows the X gantry to be slightly adjustable in terms of length.
It's not really perfect but this worked for me. I hope it helps!