MEMORY! ERROR! AAAAHHH!

Anon

Ok,

I make a bug array, anywhere from 400 - 2000 items and could be more.

Then I go through around 5 loops, one in the next,

while(){
for(){
while(){
if(){
}
}
}

I also load each item from the array mentioned above (which are file pointers) into another array, 1 at a time,
like so:

foreach($filesearch as $this){
clearstatcache();
$fpath="$rpath$this";
$infile="NOPE";
if($infile=file($this)){
$done="NO";
$that=split(".",$this);
if($that[1]==$ftype[0] or $that[1]==$ftype[1] or $that[1]==$ftype[2] or $that[1]==$ftype[3] or $that[1]==$ftype[4] or $that[1]
==$ftype[5]){

and some things I dont want to show you here,
not big items, just if statments looking for spacific things in the file.

clearstatcache();
}

and do some things with 1 line of text from each.

It bombs with memory errors every time, so is there any way to clear the used memory so far? or something like that? Or am I just stuck...

[deleted]

What on earth are you trying to do here??

Anon

you don't want to know.....

Anon

oops, that was me "the guy"

I have successfully written 3 search engines in php, and tons of people are using them with excellent results, so now I have someone who wants this to be able to get results up to 10 directories deep, which means running through all the directories, and I load the full path into an array, this is quick, fractions of a second.

Then I use the array and do my searchg on each file pointer, but its running out of memory now.

My code is very fast and efficient, but when it starts $file1=file($this[0]);
it runs out of memory.

Too many arrays?

I just wondered if I could clear all memory used that is no longer needed somehow..

Anon

What you could try is serializing some of your results or writing results to a standard text file when the array reaches a certain boundary

You could also flag the first arrays as seen display them and then unset then look at the rest

These are three options that could work together there is probably loads more

An old PERL saying... there's more than one way to do it, and that is so right

Mark
hdwt.com

P.S
You should really break your arrays searches into modular functions it's so much easier to follow

[deleted]

"I have successfully written 3 search engines in php, and tons of people are using them with excellent results,"

Judging from your codesample I'd say you have LOT to learn about search engines.

Rule #1 is that search engines NEVER do a realtime search in files. Instead, they index the files on a regular basis, and they only search the index.

Second, if you do have to search the files (which you don't) you would never ever store any data about the files other than which files match the search.

There is simply no need for nested loops, ever. Just one recursive function to read all the files one after the other.

I am very curious about how your search engine can get any sort of acceptable response (<10ms) for a search.

Anon

Well look at it like this, ever try to get a search engine for your website, and by engine I mean spyder/crawler type, for free?

Good luck, and never without ads or having the ability to customzie it.

There are only bits and pieces of the code there, you have no idea what the entire thing looks like, and there are only a couple of loops, the example I provided was a special case, for a single website.

[deleted]

"There are only bits and pieces of the code there, you have no idea what the entire thing looks like, and there are only a couple of loops, the example I provided was a special case, for a single website."

Woops, didn't mean to offend, just trying to help out. I know I have a tendency to sound arrogant and snobby and I-know-it-better-than-you-ish.

The thing is, from what I see you do here I get the feeling you are programming your searchengine the hard way, while there are much more efficient/effective methods out there.

Have a look at the 'slapping together a search engine' article in the articles section, that is a good start for a simple fast and effective searchengine.