It may be that the server is either using a protocol or cipher that IE5 does not support.
SSL has several protocols and ciphers. The client and server negotiate to find ones that they are both willing to use. If they don't have any in common, then communication is impossible.
Session key size is also negotiated. If the browser insists on no more than 40 bit, and the server insists on 128 bit, then the connection will fail.
Mark