stripping out html when searching

rakennedy75

This code searches the database and provides a link to the page that has the hilighted text. Pretty straight forward.

$query = "SELECT * FROM $table WHERE body LIKE '%".$keyword."%'";
$result = mysql_query......
while ($r=mysql_fetch_array($result)){
	echo "<a href='page.php?hilite=$keyword'>".$r['page']."</a>";
	echo "<br>";
}

The pages have html tags and attributes stored in the database. As you can see above I have a hilite variable which will be used to search and highlight the text that was used for the search..

		$body= str_replace($hilite, "<b><font color='#990000'>$hilite</font></b>", trim($r['body']));
		echo $body;

The problem I have is if the word that is being searched for might be part of an html tag and if this gets replaced than it messes up the display of the page. Example would be if someone looked for the work body, this would screw up the body tag and in result the display of html will be messed.

Is there a way I can strip out the html tags so they don't get searched? Or is there a better way to approach this?

Thanks for your time.

Weedpacket

Well, you'd probably have to take it out of the database and search using [man]strip_tags[/man] to get rid of the HTML. That would be a pain, because you'd have to take all of them out to search them.

You could store a strip_tags version in the database alongside the HTML version, and search in that, but that makes for an unsightly duplication of data.

Ideal would have been to not store the HTML mixed in with the content, but I guess that's no longer an option.

Ditto to using a dbms that would have allowed you to write an entire function that could read the field into a variable, strip the tags, and search the remainder - all before returning the result to PHP.

It's a tricky one....

Sam_Hobbs

I am not sure if I understand your question, so please excuse me if my comment is not relevant.

Could you process the HTML as HTML using the DOM? If so, then instead of the HTML being a problem, it would be part of the solution.

rakennedy75

I am still building this and nothing is currently set in stone.

The reason I am storing HTML in the field is that I might have other designers that do not know php and they can just create the page in a editor (such as dreamweaver). Drop souce code into a field and upload. Its not to late to implement other solutions. However, I would really like that to be an options for the designers.

You mentiioned something about using he dbms (I am using MySQL) to use a function to do all the work before returning a result. I would really be interested in that. Even if it wasn't for this particular use. Do you have any idea on where i can get some information on this. I wouldn't even know where to look. Does this involve the use fo regExp?

Thanks for your time

Weedpacket

Originally posted by rakennedy75
You mentiioned something about using he dbms (I am using MySQL) to use a function to do all the work before returning a result. I would really be interested in that. Even if it wasn't for this particular use. Do you have any idea on where i can get some information on this. I wouldn't even know where to look. Does this involve the use fo regExp?

Well, most major DBMSs offer the facility to write so-called stored procedures, that become part of the database, and basically allow you to write your own custom functions to use in queries. MySQL's documentation should supply the details of how it implements the things.

Originally posted by Sam Hobbs
Could you process the HTML as HTML using the DOM? If so, then instead of the HTML being a problem, it would be part of the solution.

Unfortunately, HTML is not XML. To be reliably manipulated by the DOM, either XHTML would have to be used, or the designers would have to be careful to stick to the XML-compliant parts of HTML. There would still be the problem of taking a wodge of text and searching through parts of it.

Sam_Hobbs

Originally posted by Weedpacket
Ditto to using a dbms that would have allowed you to write an entire function that could read the field into a variable, strip the tags, and search the remainder - all before returning the result to PHP.

Why is it a necessity that it be done by the DBMS? This is not a typical database function. Why can't the stripping be done by PHP using software designed for the purpose?

Why not use sofware such as a regular expression parser? Or parsing software that is more thorough and accurate than a regular expression?

Sam_Hobbs

Originally posted by Weedpacket
To be reliably manipulated by the DOM, either XHTML would have to be used, or the designers would have to be careful to stick to the XML-compliant parts of HTML.

Where is that said?

The official documentation of DOM is at Document Object Model (Core) Level 1I don't see anything saying that it applies only to XML. I do see things saying that HTML is supported by the DOM.

tsinka

Hi,

look at this thread on experts exchange:

http://www.experts-exchange.com/Web/Web_Languages/PHP/Q_21072602.html

Thomas

Weedpacket

Originally posted by Sam Hobbs
Where is that said?

The official documentation of DOM is at Document Object Model (Core) Level 1I don't see anything saying that it applies only to XML. I do see things saying that HTML is supported by the DOM.

I said "reliably" and "XML-compliant". All too much HTML is very badly written (thanks, Microsoft!) and serious (and unspecified) coding has to be done to knock it into a usable shape. Before you can use the DOM to do anything with HTML, you have to parse it into a DOM structure. Which DOM implementation are you suggesting be used to do this?

BSTRhino

The reason most people prefer using a database to parse things is because then you can query, search, sort and do everything else that a database can do with that data.

I really enjoy using relational databases to do those sorts of things, so what I would do is keep double copies of the data. One with strip_tags on it and one without. This however, would take up twice the space, so it is up to you.

Weedpacket

I dunno - maybe MySQL does have some string functions (regular expression based, probably) that could be used to remove <...stuff...> and used in a LIKE query. Or do the query with tags in place like now, and then strip_tags() what comes back, and search again in PHP.... Eeeww...

If it's that early in the project, consider (strongly) going with a more template-based approach. It's not as though every single page is a unique work of design in itself, is it? Right now it sounds like you're taking on a lot of work to yourself that could by rights be distributed a bit more fairly. But I do say this without knowing just how free-form the stuff your designers are producing is.

Sam_Hobbs

Originally posted by Weedpacket
I said "reliably" and "XML-compliant". All too much HTML is very badly written (thanks, Microsoft!) and serious (and unspecified) coding has to be done to knock it into a usable shape. Before you can use the DOM to do anything with HTML, you have to parse it into a DOM structure.

That is two different things. Badly written HTML is not the same thing as non-compliant XML. It is not necessary for HTML to be XML. HTML must be HTML and a DOM implementation might have a problem with non-standard HTML but HTML does not have to be XML.

Originally posted by Weedpacket
Which DOM implementation are you suggesting be used to do this?[/B]

This is a PHP forum. Is there a choice?

Did you look at the W3C documentation? If not, then there is no value in continuing the discussion.

Weedpacket

Originally posted by Sam Hobbs
If not, then there is no value in continuing the discussion.

Well no, not if you've got no way of applying the suggestion.

Sam_Hobbs

Originally posted by Weedpacket
Well no, not if you've got no way of applying the suggestion.

Note that I am trying to help, just as you are. I am trying to ensure that the OP (rakennedy75) gets the best help possible. Please don't stop trying to help rakennedy75 because of me. However I just think that XML is given too much attention in situattions such as this; that the DOM is not for XML exclusively. So now that I have said that, I will let rakennedy75 decide about that.

I don't know much about XML but as best as I can see, it is like a database. So perhaps the HTML can be stored into a XML file and processed as part of the XML file. That might provide the best of both.

rakennedy75

Thanks everyone for contributing. I hope my post wasn't the result of causing an argument but this is the proof that their are multiple of ways to approach a solution and I appreciate all the help you have given me. Regardless of which way I choose to go on this you have opened my eyes to something I had no knowledge of. You have also created a lot of work for me since I want to research everything you have brought up.

One solution I played with was creating a separate database that held all the html attributes (with three of more characters) I could think of. I would use that and check it against the keyword used to search and not allow a result if it matched. Along with this I would only allow them to search three or more characters. Does anyone on here have any thoughts on this solution. I can than use this database for multiple sites. Just something I was thinking of doing.

Thanks again for your time.

Sam_Hobbs

I am unclear about what you are saying so I can't answer very clearly.

Would it be a problem to determine if a match were an attribute for a tag or just part of text outside of a tag?

Would comments be a potential problem? I often comment out HTML (as in ) but that might not be relevant to your situation.

rakennedy75

Thats stuff I would take into consideration when doing it, I am sure this is not the most efficient solution.

Basically I would have table where I would place tags or characters that might be with in a tag over three characters. For example these are some words I would put into a table.

html
htm
php
body
border
cell
table
font
style
class
meta
http
size
and so on ...

It would pull this list from a database and check them against the search keyword and than return a message to the user saying couldn't search keyword if it found a match. Or it just wouldn't place the highlight tags around it.

So if someone typed in "html" or "table" it wouldn't return the search. Or I could simply just not add the highlight tag since that is what is really messing up the display. Only on specific keywords found in the database.

The draw back is if this end up being a website where someone wanted to search for results of the word, "body" or "border". I wouldn't think it would be that big of a deal but something to think about.

Like I said it was just a thought and wanted to get feedback on it. It might be a lot of work to add all these attributes and html tags but I have some control over this since the only people who would be adding new html tags are designers that I work directly with.

Let me know if this makes sense.

Sam_Hobbs

First it would help us understand if common terminology is used whenever possible. For example, html, body, table and font are all tags (right?) and style is an attribute. php is actually a processing instruction tag; they are "<?" and "?>". It is possible for languages other than php to be in processing instructions. There is no such thing as "htm", "cell" and "http" in html, as far as I know. The "http" is a protocol that is part of an address (URL).

So are you saying that you just need to make a list that is the equivalent of a list of reserved words? Perhaps that would be easy enough to do and to use. Unless a tag or an attribute is also a word they need to search for.

It is easy to get a list of all tags and attributes. However the DOM already knows what they are and you can use it to search in the text. You can have the DOM find for you each tag and for the ones that have text that should be searched (such as "<p>" tags), you can get the text in them and search it. If that can satisfy your requirements, then it is more elegant in the sense of being more convenient for you and the user and it is essentially perfect in it's accuracy.

rakennedy75

Makes more sense that way, I will take a closer look at that and see what I come up with. Sorry about the confusion. I should have been more clear with the tags and attribute terms. As far as using the word "cell" I was trying to demonstrate that a shorter version of an attribute like cellpadding would be affected as well. And the "http" would be a common concern when an absolute url link is used in a (<a href...) tag. I doubt I will use this approach, just wanted to get some ideas on what you guys thought. Thanks for taking the time. You have been a great help.

tsinka

Hi,

find attached a script that worked on my server. It takes care of

PHP-Code
all HTML tags
URLs
javascript blocks
object blocks

and it only replaces search words inside the body tag.

Might need some tweaking but it is a solution without using a database or a list of reserved words.

Thomas