Yes this sounds strange! If not tapered and no flange or head does it bottom out in the hole? Otherwise what would stop it from baking out or working it's way in?How about a photo so everyone can see what you're looking at..?
A bit of context about the machine would go a long way to help as well.