Technically at that point you're not your own ISP, rather your own hosting company :p I run a home server, but I don't give out access to everyone. Rather I use it mainly for testing and put what works to my "live" server.
Over FiOS it's not bad, cable is a little worse.
<tirade>We have our own dev server at work that we give client's access to view over Comcast's Business Class internet, and they are horrendous. Our internet dropped out for the entire time i was doing my working interview there (try developing an OOP blog starting from scratch with absolutely no access to resources online, and nothing in print that is extremely helpful). Ever since then, if the weather turns bad, we lose internet (like this coming weekend, or the past few days). Even if the sun shines in such a way, our internet slows down.</tirade>