As long as you only have integers in the varchar(10) field, you should be OK.
Now, that said, I always take precautions and make a copy of the table and change the datatype on it. I have performed this in two different ways: copying the table, then altering the datatype, or creating a brand-new table from a SQL script that converts the datatype for me. Whatever works for you.