Ok guys, I got this working. This article will explain a lot:
http://codespatter.com/2008/02/19/how-to-break-a-mysql-left-join/
Basically, writing the where statement (or the way Lars suggested to add additional conditions to the on statement) breaks the Left Join.
The key part of the fix for me was to use this option (WEEK <=10 or WEEK IS NULL) along with a having condition.
Here is my final fix:
SELECT t.TEAM_ID AS TEAM_ID, TEAM_NAME, TEAM_LEAGUE,
COALESCE(SUM(WIN),0) AS WIN,
COALESCE(SUM(LOSS),0) AS LOSS,
COALESCE(SUM(TIE),0) AS TIE
FROM teams t
LEFT JOIN team_stats s
ON t.TEAM_ID = s.TEAM_ID
AND (WEEK <=10 OR WEEK IS NULL)
AND (SEASON = 4 OR SEASON IS NULL)
GROUP BY t.TEAM_ID
HAVING TEAM_LEAGUE = 4
Thanks all!