The L brackets that attach to the servo mount plate and the skull did not fit properly inside the skull. The screw holes are too close to the edge of the plate. When the L brackets are mounted to the plate and turned so they face the skull edge, they extended past the edge of the plate a little. The plate is already a tight fit, and with the brackets not quite fitting correctly, the plate would not fit properly into the skull. It would push the sides out a little and then the top of the skull would not fit properly.
To fix this I simply turned the L brackets around the other way and used a longer #4-40 bolt to reach the threads.