Yup, you need to use timestamps. Look in the Date/Time Functions section of the manual for more info..... it's very well explained there.
$numDays = 5;
$then = mktime() - (86400*$numDays);
echo date("Y-m-d", $then);
BTW, 86400 is the number of seconds in 24 hours.
-geoff