Efficient Dir read - Lots of files??

KendersPlace

Hello there. I am fairly new to PHP, but have been coding ASP for several years. I have an application with some special needs and was looking for some advice from some of you w/ more experience.

Basically, I'm writing an app for a linux box that is a storage server for security camera feeds. The camera feeds are stored as multiple individual .jpg files, each about 7k in size. I need to write a remote retrieval app for this box using PHP.

The user will specify a "start time" and "stop time" and the php will grab the images out of the directory that fall within that time range. It will then loop through the list to add them one by one to the resulting page, or add them to an array. That part is easy. The part I need some guidance with is the retrival and examination of file names and times.

Each image is time stamped as a part of the name. Example Img_20030205153023.jpg

I've done this with success on a small scale, however, in this case... we're going to be talking about a LOT of images. Each dir will have about 100,000 to 150,000 files in it.

I am going to use X-FS file system as I hear it is extremely fast for lots of small files. I was going to open the directory object in php then let the php scan through doing a one by one date comparison with the requested query string... which will work, but this can't possibly be very efficient.

Will this process crawl (dunno, haven't tried it yet), or is there a more efficient direction I can start out in?

Thanks!!

Weedpacket

I think whatever solution you end up with will boil down to this. Open the directory, read the filenames therein, and find those that match the requirements.

You might be able to save time by reading the file's change time attribute (via filectime) from the filesystem and seeing if that falls within the desired range - that may be faster than parsing the filename itself (you'll nead to read it from the directory anyway, of course).

ednark

php has functions which drop you to the command line...
read up on the [man]exec[/man] functions.

it may be faster to just grep the directory or use some other native unix program to do the searching for you... then read in the results of the execution and do you thing

however if you are going to take any form data of any sort and use it to drop out of php into the commandline be very very very very very very carefull about validating the data first