Two things have to happen to make it work, battery has to come in and it also has to go through the bulb and go out.
With your test, it is saying the battery is getting there and the path is good while you are holding it.
So the big question has to be why the path is NOT good if you are not holding it?
Are the contacts in the bulbholder not making solid contact when not holding the bulb?
Is the ground good while holding and not when loose? Tricky sometimes if we are moving the base a little bit so that a mounting screw that gives the ground gets contact but not when we turn loose!
Look at the contacts and make sure they are clean and if they are the little tabs that stick up, make sure they are not bent down too far to make the connection.
But check the bulb for sure. If you have a meter, check continuity from one contact to the other. If no meter but some wire, touch one wire to negative and the other contact to positive on a battery and see if it lights! A little spark but nothing dangerous if you keep the fingers back enough to not burn. Or swap it with one you know is working?