Do you have a stop nut on the thread, or is it stopping when it bottoms in the hole? If it's the latter, add a thin nut that will stop it going in quite so far. Adjusting the nut will also control the angle it stops at, and tightening the nut when it's in will lock it place so it doesn't vibrate out.