If wax has cooled, try these techniques for removing it:
If the candle holders are waterproof, run them under hot tap water and remove the wax by hand.
Wax will shrink after a few hours in the freezer, making this a good technique for cleaning votive candle holder. The wax inside should pop right out. Any remaining residue should come off with just a light washing with mild dish soap.
In order to avoid wax from sticking in the first place, fill votive candle holders with 1/2 of water before placing in the candle.
(Source: Martha Stewart Living)