Problem with capturing something with a regular expression

andarian

Hey all

So I have a script that crawls certain websites, brings the content and strips out, among other things, links tags entirly.

I have a preg replace with

  '@<a[^>]*?>.*?</a>@si',

That does the job. Everyone were happy for some time, till some pages did problems. Eventually I found out those pages have something like

<a href="whatever">whatever</a></a>

This pretty much freaks the rest of the script and things gets striped out randomly.

I have a pretty weak control in regexes, I tried to change the regex so it handles this kind of situation, but so far to no avail.
Any help would be greatly appreciated!!!

sneakyimp

'@<a[^>]*?>.*?(</a>)+@si'

andarian

works nice!

how about if I have something like

<a href="whatever">whatever</a>whatever</a>

I know this looks like broken HTML but some sites I'm crawling actually have this...

sneakyimp

that's more difficult because 'whatever' might be other links. if there are no html tags in 'whatever' then this might work...no promises.

'@<a[^>]*?>.*?(</a>([^<]*))+@si'

andarian

OK I get it.

But I cant I do something like

'@<a[^>]*?>.*?(</a>([^<a]*))+@si'

Which is almost exactly what you wrote,
Probably I didnt do it right, but the idea is to look for

 <a>something</a>something as long as its NOT "<a" and now an optional </a>

sneakyimp

I dunno. I suppose you'll just have to test it. Personally, I wouldn't worry about broken HTML.

andarian

hehe, why not worry?
I have a script that crawls pages and reads the text in them. for that I have to first strip out tags. But with broken html like that, what happens is that sometime the strip functions strips away the entire page