You /must/ use reference. No one ever gets good at drawing hands without looking at a lot of hands. In plenty of drawings, I don't use reference, either because I have a pretty good mental image of what the hand looks like from a particular angle (this comes with time from practice and observation) and/or the hand is not an important element of the picture -- but if I'm ever unsure, or if the hand is a big part of the picture, I always look in the mirror or take a photo or, if the other two options aren't available, just look directly at my hand as best I can. Or someone else's.
Hands are everywhere. Most people have two of them with them at all times. There's never a good reason for not using a reference when a hand becomes a drawing challenge.