Ok. I've been looking around and I've been trying. One solution, people seem to opt for is deleting and creating the cookie.
// create the new one
setcookie("testcookie", "NewValue");
// delete the old one
setcookie("testcookie");
Important here is the order, which is counter intuitive. First set the new cookie and the second setcookie will delete the old on, because they seem to be handled in the reverse order.