Efficient coding (help needed)

PaPPy

I need to make this code more efficient and speed it up.

It is ran through command line.

It imports XML thru $vulneratorxmlfile
it looks at the Affected column and creates duplicate entry in the output file for each computer name.(sometimes it is an IP address)

It also looks up in the csv for the active directory OU path.

It also looks in another csv for the IAVA to TCNO mapping.

All this gets saved into a new excel XML workbook.

Right now, on a small 2mb $vulneratorxmlfile xml file, it is taking about 7mins and 30mb of to output the file.

I have included the php and the csv&xml files in the attached zip. I tried displaying the php code, but went over the length

So if there is anything that can be done to make this more efficient, faster, streamlined. I would appreciate it.

Thanks

PaPPy

also i have noticed it is faster to run this code on my computer. than on a xeon server with 16gb of ram.

how does that make any sense?

NogDog

PaPPy;10984341 wrote:
also i have noticed it is faster to run this code on my computer. than on a xeon server with 16gb of ram.

how does that make any sense?

Depends on what else the server is doing at the same time. (Probably also depends on number of CPUs and how fast they are.)

PaPPy

server isnt doing anything 🙁

NogDog

One thing I might try is, instead of calling the getcsvvalue() function like you do within the foreach loop, instead initially read the CSV files into arrays, and then use a function to search those arrays. I would think that would be quicker than constantly opening and parsing the CSV files. Also, you might find reading them in via the fgetcsv() function to be more efficient, e.g.:

/**
 * Read a CSV file into a 2-D array
 * @return array (false on failure to read file)
 * @param string $file pathname of CSV file
 */
function csv2array($file)
{
   $data = array();
   $fh = fopen($file);
   if($fh == false) {
      user_error("Unable to read '$file'");
      return false;
   }
   while(($row = fgetcsv($fh)) !== false) {
      if(count($row) > 1) {
         $data[] = $row;
      }
   }
   return $data;
}

The caveat here would be if the CSV files are really large, then you'd be using up memory with the arrays, so it may be a balancing act and just have to see which works better.

PaPPy

had to make a small change to your code
$fh = fopen($file, "r");

but as my csv is 2 columns the array_search doesnt work

Array
(
    [0] => Array
        (
            [0] => IAVA
            [1] => TCNO
        )

[1] => Array
    (
        [0] => 100
        [1] => 200
    )

any suggestions?

NogDog

If it's just two columns, you could do something like:

   while(($row = fgetcsv($fh)) !== false) {
      if(count($row) > 1) {
         $data[$row[0]] = $row[1];
      }
   }

This assumes the first column ($row[0]) is unique. If the inverse is true, then just flip the indexes. Then you can simply check for that index when using that array later.

PaPPy

had to flip the 0 and the 1
$data[$row[0]] = $row[1];

but will try and see how this speeds things up

PaPPy

I also noticed it works fine, but for my csv that has computer_name, path/of/computer

the slashes seem to throw things off. if I replace the slash with text, or pipes it works fine.

extremely strange

NogDog

PaPPy;10984542 wrote:
I also noticed it works fine, but for my csv that has computer_name, path/of/computer

the slashes seem to throw things off. if I replace the slash with text, or pipes it works fine.

extremely strange

Is it the fgetcsv() that seems to have an issue with it, or something about the resulting array that is a problem? (I can't off the top of my head think why it should be a problem in either case, but sometimes I just can't think, period. 😉 )