AJAX!!!!!
Well, more specifically, the entire interface is done in AJAX. The spell-checker is done through JavaScript (well, it may utilize some components of AJAX for the dictionary; obviously downloading an entire dictionary to a user's computer would be impractical).
So, to answer your question, I'd go with JavaScript. JavaScript also takes care of the nice formatting and rich-text capabilities; http://www.htmlarea.com/ will help you if you're looking for them. They're called WYSIWIG text editors, and they're quite useful in certain applications (E-mail editing, newspaper management, etc.).
The spell-check is also done through JavaScript, but, like I said earlier, it utilizes some components of AJAX to download (parts of) the dictionary to the user's machine at run-time. There are a lot of free JavaScript spell-checkers out there as well.
Obviously both Gmail's WYSIWIG editor and spell-checker have been custom-built; if you're not that advanced with JavaScript, take a look at some of the already-made solutions at that website or do a Google search (ironic? 😃)