From the manual:
"Once a slave is properly configured and running, it will simply connect to the master and wait for updates to process"
As near as I can tell, MySQL's replication is based slaves looking at the logical log of the master. As soon as the master has updated the logical log, the slaves read it and copy the updates.
"I was envisioning that an update may be propagating while the slave is receiving a query. What happens in this case?"
I guess with MySQL's rigid locking mechanism, the tables in querstion will be locked during the update. You'd have to give this a whirl on a very large query, see what happens.
"The root of my question was trying to determine in which direction communication is initiated for replication. From your answer to the first question it appears that the master tells the slaves when an update has occurred"
Not quite. If you read the manual a bit, it says that the slaves connect to the master, and keep that connection open. They look at the master's binary log. As soon as that changes, the slaves read it and process the changes. So the master is not even aware that there are slaves listening.
"Is there an associated communications overhead"
Yes there is, but wether that is a problem depends on your setup.
You may want to read the replication FAQ:
http://www.mysql.com/doc/R/e/Replication_FAQ.html