Don't sweat the short term loss. Look at the historical charts for gold and silver. It always goes up. Always. Not to mention it's always been used in industry and is also collected as "money".
Silver will be fine. Hold what you have and keep buying.