Well, that is a problem, but it has to do with SQL, not PHP, though it is due to the PHP code. steaky is using variables in single quote delimited strings. Consequently, the variables are not interpolated and so the SQL statement does not have the intended values, but the strings '$userId' and '$password'. By using concatenation as you have described, this problem would be fixed.
On the other hand, another problem would appear: since $userId and $password are from incoming variables, they need to be escaped, but have not been escaped. As such, the script is vulnerable to SQL injection.