Better way to do this?

jebster

Is there a better way to do this:

$query = 'Free Tools asdfgae';
$query_exploded=explode(" ",$query);

function you_searched_for($query_exploded) {
   for($i=0;$i<count($query_exploded);$i++){
      $url = "http://dictionary.reference.com/search?q=$query_exploded[$i]";
      $file = fopen ("$url", "r");
      while (!feof ($file)) {
         $line = fgets ($file, 1024);
         if (eregi ("(.*) entries found for <i>(.*)</i>", $line, $out)) {
            print "<a href=\"http://dictionary.reference.com/search?q=".$query_exploded[$i]."\">".$query_exploded[$i]."</a> ";
         } elseif (eregi ("No entry found for(.*)", $line, $out)) {
            print "$query_exploded[$i]";
         }
      }
   }
}

This seems to take a long time to load the page! Sometimes it exceeds the maximum limit of 45 seconds even!

jeremuck

For each search term, you have to ask for another request... you may try using a user agent such as snoopy (found at sourceforge, I think). It may be able to retrieve the document a bit quicker. If Dictionary.com had a soap service, that would be quicker of course. You could just give them a link to the search term, instead of retreiving it for them.

You might optimise it a tiny bit by figuring out which area of the retrieved document is more likely to have content you want, and skipping over the areas that don't... for instance skip the first 5 - 6 lines and then start searching. But I think the benefit from this would be minimal.

jebster

Originally posted by jeremuck
For each search term, you have to ask for another request... you may try using a user agent such as snoopy (found at sourceforge, I think). It may be able to retrieve the document a bit quicker. If Dictionary.com had a soap service, that would be quicker of course. You could just give them a link to the search term, instead of retreiving it for them.

You might optimise it a tiny bit by figuring out which area of the retrieved document is more likely to have content you want, and skipping over the areas that don't... for instance skip the first 5 - 6 lines and then start searching. But I think the benefit from this would be minimal.

How would I skip lines? And I looked at there site and I don't see any "SOAP" service.

tsinka

Hi,

we had similar problems so we tried the curl functions of php and the scripts ran up to 20 times faster than by using fopen.

Did you check how long it takes if you request the dictionary page manually with your browser ?

PHP has to be compiled with curl support.

jebster

Originally posted by tsinka
Hi,

we had similar problems so we tried the curl functions of php and the scripts ran up to 20 times faster than by using fopen.

Did you check how long it takes if you request the dictionary page manually with your browser ?

PHP has to be compiled with curl support.

Whenever I try to reinstall PHP my scripts don't work! So I guess that isn't an option! 🙁

Can anyone tell me how to skip lines like metioned above? 🙂

tsinka

Hi,

if you dont have curl support in your php installation and you are running on linux/unix you might try lynx to get the contents:

I tried that url. It takes rather long until the site returns a result if nothing could be found. if your query array is very large this will result in the execution limit being exceeded. There's nothing you can do about that than increasing the time limit.

<?PHP
$query = 'Free Tools asdfgae';
$query_exploded=explode(" ",$query);

function you_searched_for($query_exploded) {
   for($i=0;$i<count($query_exploded);$i++){
      $url = "http://dictionary.reference.com/search?q=$query_exploded[$i]";
      ob_start();
      passthru("/usr/bin/lynx -source $url");
      $content = ob_get_contents();
      ob_end_clean();

  if (strstr($content,"No entry found for")) {
     echo "$query_exploded[$i]<br>\\n";
  } else {
     echo "<a href=\\"$url\">".$query_exploded[$i]."</a><br>\\n";
  }
  flush();
   }
}

you_searched_for($query_exploded);
?>

Since the site seems to support a query about all search terms you can try:

<?PHP
$query = 'Free Tools asdfgae';

function you_searched_for($query) {
      $url = "http://dictionary.reference.com/search?q=".urlencode($query);
      ob_start();
      passthru("/usr/bin/lynx -source $url");
      $content = ob_get_contents();
      ob_end_clean();

  if (strstr($content,"suggestions found")) {
     echo "<a href=\\"$url\\">".$query."</a><br>\\n";
  } else {
     print "$query_exploded[$i]<br>\\n";
  }
  flush();
}

you_searched_for($query);
?>

If you do not have lynx try the above with fopen again. It should be faster then requesting the url for each search word separately.

I tested the above code and it worked for me.

jebster

Cool thanks! I'll try those out! 🙂

jebster

Originally posted by tsinka
If you do not have lynx try the above with fopen again. It should be faster then requesting the url for each search word separately.

:o Umm how do I do that LOL! Sorry...

tsinka

Hi,

this one should work:

<?PHP
$query = 'Free Tools asdfgae';

function you_searched_for($query) {
      $url = "http://dictionary.reference.com/search?q=".urlencode($query);

  // php >= 4.3.0
  $content = file_get_contents($url);

  // php < 4.3.0
  //$content = implode ('', file ($url));

  if (strstr($content,"suggestions found")) {
     echo "<a href=\"$url\">".$query."</a><br>\n";
  } else {
     print "$query_exploded[$i]<br>\n";
  }
  flush();
}

you_searched_for($query);
?>

But be aware that it takes some time for the Site to return results so you might need to increase the execution time limit (the above example needs about 10 to 15 seconds until the Site returns the results).

jebster

Thank you very much! 🙂