Using assertions is as easy as the docs say, as long as you remember that they can't have variable length. But since that isn't needed here anyway, there really is no problems at all
# negative look behind assertions look like this (read as: pattern not preceeded by)
$pattern = '(?<!';
# You don't want to capture when it's on the form
# [url=http://example.com/file.ext]link text[/url]
# but you do want to capture when it is simply
# [url]http://example.com/file.ext[/url]
# So, when pattern is not preceeded by [url= and [ is a special pattern character that needs escaping
$pattern = '(?<!\[url=)';
# Which using your original pattern
# /(http:\/\/[^\s]+)/
# would give
$pattern = '/(?<!\[url=)(http:\/\/[^\s]+)/';
And the rest is just feedback for improvement
# and then we change opening and closing delimiters from / so we don't have to escape those
$pattern = '#(?<!\[url=)(http://[^\s]+)#';
# and then we get rid of your parenthesis, which is a capturing subpattern, since you don't use
# it anyway - you used the whole pattern, which is the exact same thing when the entire pattern
# is also a capturing subpattern.
# do note that the first set of parenthesis is NOT a capturing subpattern. It is an assertion and
# requires the parenthesis
$pattern = '#(?<!\[url=)http://[^\s]+#';
# And finally we get rid of your use of preg_match followed by preg_replace, since you can use $0
# in your replacement text to refer to the captured pattern when using preg_replace
$replace = '[url=$0]$0[/url]';
# and now you do all the work in one reg exp pass, instead of doing it in two as before
# And should you ever find yourself doing one _match followed by _replace, also capture
# the offset with match, PREG_OFFSET_CAPTURE (see docs for preg_match)
# and then use ordinary string replacement at that offset, since you now know exactly where
# the text is and what the text is, so you no longer need any kind of pattern matching.
$s = preg_replace($pattern, $replace, $search);