The index of an associative array is nothing more than a [man]string[/man], so the basic rules of strings still apply here - either use concatenation or variable interpolation (within strings delimited by double quotes).
EDIT: Also note that I wouldn't suggest using your approach above at all; why not just make $SESSION['car_items'] a numerically-indexed array? That way storing a new item in it is as simple as:
$_SESSION['car_items'][] = $new_item;