array_search looks in the values of the array, not the keys, for a match. If $SESSION['cart'][$id] had a value of say "blue", then your unset call would be:
unset($SESSION['cart']['blue'])
instead of your expected result. Since you know the ID, and the key you're trying to unset is the ID, just use:
unset($_SESSION['cart'][$id]);