Html grab, ideas

eviljosh

So i have this fairly yucky program to write that involves getting prices from from other websites on the net and then displaying them on the local page.

I have a fair amount of programming knowledge so this is what I have come up with as a plan so far, it would be great if someone with a massive amount of php knowledge could let me in on some tips.

The main function would be grabbing the prices which should be pretty easy since they do not change places on the websites. Although the problem with this is there is multiple prices on some sites so you would seemingly have to have multiple functions per site?

So after that the variables filled with prices would be thrown into a DB and then displayed on our website in specific places. So yeah what do people think?

kevinsequeira

i guess essentially what u will be doing is to read the page from the other sites and then do a search for a particular expression or html code which indicates where the price is.

happy coding

reg
kevin

eviljosh

Any idea if it would be possible to grab the beginning of a string that was unique but then instruct the code to grab "x" amount of spaces of html after the initial string?

leatherback

Yes it is.

Have a look at the string manipulation functions in the manual, try a bit of coding, and get back to us when you are stuck.

I believe the functions you are looking for are substr() and str_pos()

pyro

how about with regexp?

eviljosh

Im lazy, I was working with one of the other scripts on these pages that someone wrote to extract just strings of data. The problem was that you have to use unique beginning and ending strings, so many of times if theres nothing unique your gonna be sucking out junk you dont want. So I just made it so that the ending string doesnt display so that you can select as much as you want to make it unique. A bit weird but its seeming to work.

trooper

Aside from the techincal issue here is there any concern for the legal issue ?

Are you permitted to actually do this ?

Just an annoying question that i felt i had to ask 😃

juschillinnow

could be big legal issues, maybe. Do you think that Burger King actually gets McDonalds permissions to compare prices?

But what you're doing could infringe on some technical issues.

Since you cannot really copyright prices it should be ok to quote them. But you may not be allowed to put the websites name next to it to compare prices (which is what I'm assuming you are doing).

As long as you have no direct reference to the sites....Who's going to question it and who could prove it?

DeltaRho2K

From a legal standpoint, if one wanted to, they could probably do some monitoring to see that you are pulling data from their site. I don't think there is too bad of a legal issue with this, but be careful, and let them know what you are doing. (Cover your a$$) If you don't CYA, then you open yourself up for problems in the future.

Now, the other thing is this: What if their data is actually database driven too?!? I mean, if you searched their actual page prior to the php, asp or whatever parsed the document, this could screw up the results... (Just a thought)

eviljosh

Thats quite true. Although I will not be putting it on a timer, the website admin will just bring up a page with a button on it and when you click the button it would update all of the pricing. In theory they could go look at the page and make sure everything looked ok, if it didnt they could just click the button again.

As for legal issues, even though the sites are not local they are affiliates so this is something they are in on also. Either way I dont own the company lol.

But hey thanks for the ideas and input any other ideas im definately open to.

The funny thing is I just recently switched my major from CS to History because I wanted to teach, I never thought id be doing jobs like this again!

eviljosh

Ok so I have this snippet of code, it is fully functional although id really like to strip the > out of the variable espescially when it goes into the db. How would I do this too ensure that when it goes into the db and then is displayed from there, the "<" or whataver character I dont like wont be there?

<?php

$grab_url = "";
$lines=file($grab_url);

// set up the start and end strings
$req_str=">$23";
$end_str="</font>";
// then scan each line for the required string

unset($the_str);
for($loop=0; $loop<count($lines); $loop++ )
{
if ( stristr($lines[$loop], $req_str) )
{
//got the start string now need to extract the bit we want
$startbit=strstr($lines[$loop], $req_str);
$endbit=strstr($lines[$loop], $end_str);
$str_pos=strlen($lines[$loop])-strlen($startbit);
$bit_len=strlen($startbit)-strlen($endbit);
$boneci=substr($lines[$loop], $str_pos, $bit_len);
break;
}
}

echo "$boneci";

juschillinnow

if you want them to show up late then you could use htmlentities or htmlspecialchars to encode them to html

or do a str_replace(">", "", $yourstring);

eviljosh

str_replace doesnt seem to cut the < out before it goes into the db, is it supposed to or should I use something else?

juschillinnow

str_replace, should replace about every character. try escaping the character like str_replace(">", "", $whatever);

eviljosh

Thanks for all the help guys, I definately appreciate it. Although I have another question. I am trying to update multiple rows with the same query, is this not possible? It seems if I stick a couple things in there to update itll work but more than one it doesnt want to. CHeck out my code for this.

<?php
// set a variable with the database name

$database_name = "_websites";

// attempt to connect to database and store result

$dbh = mysql_connect("localhost","dude","admin");

// check the result of the connection attempt

if (!mysql_select_db($database_name)) {

echo "Can't Select $database_name";

}

$sql = "UPDATE prices SET price='$mareci' WHERE name='ez' AND brand='m'";
$sql = "UPDATE prices SET price='$cameci' WHERE name='ez' AND brand='cel'";
$sql = "UPDATE prices SET price='$wineci' WHERE name='ez' AND brand='wn'";
$sql = "UPDATE prices SET price='$saleci' WHERE name='ez' AND brand='s'";
$sql = "UPDATE prices SET price='$pareci' WHERE name='ez' AND brand='pt'";
$sql = "UPDATE prices SET price='$keneci' WHERE name='ez' AND brand='kt'";
$sql = "UPDATE prices SET price='$beneci' WHERE name='ez' AND brand='Bes'";
$sql = "UPDATE prices SET price='$paleci' WHERE name='ez' AND brand='pll'";
$sql = "UPDATE prices SET price='$luceci' WHERE name='ez' AND brand='l se'";
$sql = "UPDATE prices SET price='$cheeci' WHERE name='ez' AND brand='chld'";
$sql = "UPDATE prices SET price='$phieci' WHERE name='ecz' AND brand='phis'";
$sql = "UPDATE prices SET price='$roteci' WHERE name='ez' AND brand='rons'";
$sql = "UPDATE prices SET price='$lmseci' WHERE name='ez' AND brand='M'";
$sql = "UPDATE prices SET price='$duneci' WHERE name='ecz' AND brand='ill'";

mysql_query($sql);

DeltaRho2K

You will probably want to run your query in a foreach loop...

The way you have this written now, it probably only does the first update.

eviljosh

Ive looked at the foreach funtion and am still having problems figuring out how to implement it since the examples are usually not used within the context of mysql update statements. Not to trouble you but is there anyway you could give an example.

DeltaRho2K

Well, the way I see it, you need to assign your variables to an array, then use a foreach or for loop to update each record. I really can't give you an example, because I have never had a need to do this, but this post may help point you in the right direction:

http://www.phpbuilder.com/board/showthread.php?s=&threadid=10209125&highlight=update+loop

Apparently, updating multiple rows can get quite complicated, so good luck. You may want to repost this portion of your question as a new thread, to attract the "multiple update gurus".

batman

Try putting a period before the = sign. e.g.

//you don't need to on the first one
$sql = "UPDATE prices SET price='$mareci' WHERE name='ez' AND brand='m'"; 
$sql .= "UPDATE prices SET price='$cameci' WHERE name='ez' AND brand='cel'";
$sql .= "UPDATE prices SET price='$wineci' WHERE name='ez' AND brand='wn'";
$sql .= "UPDATE prices SET price='$saleci' WHERE name='ez' AND brand='s'"; 
$sql .= "UPDATE prices SET price='$pareci' WHERE name='ez' AND brand='pt'"; 


//then run the query
$query = mysql_query($sql);
if(!$query)
{
echo "I'm sorry, it did not update correctly.";
}
else
{
echo "Update successful.";
}

DeltaRho2K

I didn't even think to concatenate the sql statements like that. I guess I was thinking that eviljosh was dealing with a varying amount of update queries (based on his earlier posts of what he was actually trying to do.)

Good catch Batman.