Yes they can. no, performance is not generally faster than int4, since 32 bit machines are the most common, and int4s are only 32 bits, they're a good match for performance.
Now, on 64 bit machines, big int (i.e. int8) is probably as fast or faster than int4, since it's the native format.
The real issue with performance is index matching. Postgresql (since 7.2 I believe, maybe just 7.3) no longer autocoerces types from int4 to int8 and back. The default type for integers is int4, so something like:
select * from table where row_id=12345;
where row_id is an int8 will NOT use an index on row_id automagically, since the 12345 is assumed to be an int4. You have to cajigger the planner right now to make it work right:
select from table where row_id='12345';
select from table where row_id=12345::int8
etc...
I don't know if this is on the table for getting changed for 7.4, but it causes issues with fk mismatches for newbies alot (i.e. they make a table with an int8 for the serial type, but the subordinate tables are int4 or vice versa).