To answer my own question, I couldn't set up sql_mode for individual database, but I can set it up for inidivdual session.
http://dev.mysql.com/doc/refman/4.1/en/server-sql-mode.html
Here is the manual for mysql 4.1 about sql_mode.
My new question is that it seems that if I am on mysql 4.1, I have no way to set the mode to be strict_mode like mysql 5 default set up?
So insert / update will silently change the "invalid value" into "closest" value, like empty string or 0 for null? Even it is what I want in this project/database. But I don't want this sql_mode in other projects and other database running on my mysql 4.1 databse server.
Or there is a way that in mysql 4.1, that I can set sql_mode to strict_mode?
Thanks!