The "I've done a funky optimization" thread

bubblenut

A thread for when you'ce got that nasty sluggish little bottleneck into an tarmac smouldering speed machine, or something similar.
As I'm sure you can guess, the reason I've started this thread is because I've done one such thing. It wasn't particularly slow to begin with but but now it's particularly not slow. It's a function to find what letters are in a string and count the occurence of each (I use this kind off thing for my fuzzy search stuff).
What I had originally looks a little sloppy but I couldn't think of any other way of doing it.

function getLetters($string)
{
  $letters=array();
  for($i=0;$i<strlen($string);$i++) {
    if(array_key_exists($string{$i},$letters)) {
      $letters[$string{$i}]++;
    } else {
      $letters[$string{$i}]=1;
    }
  }
  return $letters;
}

But I managed to speed it up by an order of 10.24 (a mean taken over 100 iterations of a string 322000bytes long)

function getLetters($string)
{
  while(strlen($string)>0) {
    $letters[$word{0}]=count_chars($word,$word{0});
    $word=str_replace($word{0},'',$word);
  }
  return $letters;
}

Any ideas optimize this further would be great.
Hope to see more Speedy Gonzalez(sp?) bits of code.
Bubble

mtimdog

Sounds like you're trying to do something similar to [man]count_chars[/man]

bubblenut

Oooh! count_chars is probably going to be a little quicker than substr_count isn't it. I've edited in the above listing.
NB, the code above does do a little more than just count_chars. Count_chars doesn't find what characters are in the string only how many for any given character.
Bubble

mtimdog

Actually, it does find what characters are in the string.

Example:

<?php
$str="aabbdd";

$chars=count_chars($str, 1);

?>

$chars now holds
(let's say the ord(a) is 65...if I remember correctly)
'65' => 2,
'66' => 2,
'68' => 2;

so, you could loop through the array and output the
chr(key) and the value.

mtimdog

Also, it looks like the php man meant to say

$result = count_chars($data, 1);

instead of

$result = count_chars($data, 0);

bubblenut

:o :o :o Guess I should have actually read the man entry first eh? Oh well, now I guess this particular example's been optimized about as far as it'll go then from a rather sloppy basic iteration/count function to a quicker and tidier version to a build in function all in less than a day. If it weren't for the thick layer of mud on my face I'd feel top of the world :p
Bubble

mtimdog

I never knew that function existed until today either:p

Moonglobe

well this may sound obvious but never keep more than you need in memory... i rewrote my bzip script (cuz im on windows and im too lazy to use anything but the built-in .zip functions) from reading the entire file into memory and the writing it out, to reading a K, writing it, and looping. It was a lot faster, tho i don't have any real guess on the speed, i dont use it very much as i still need an untar program for most bzip files 🙁

bubblenut

How do you deal with regexs like that? What happens if half of what you're trying to match lands in one chunk and the other in the next?
Cheers
Rob

Moonglobe

wha?😕

bubblenut

Sorry, I'm having difficulty actually getting out what I want to say today. So far I've been managing to get it right on the second attempt so we should be OK now.

If you're reading a file in 1k chunks and searching for something in that file, how do you get around the problem that half of that something may be on one chunk and half on the next chunk. Ie, there is a mtch in the file but it won't be found my the search because it only sees the first half and then the second half seperately?

here's an example file

<example>
  <character id="1">
    <fname>Barney</fname>
    <sname>Rubble</sname>
  </character>
  <character id="2">
    <fname>Fred</fname>
    <sname>Flintstone</sname>
  </character>
</example>

Now, if we were to read this file in three byte chunks then none of the tags would ever be matched. We'd have to read in at least 9 bytes to match the first tag and to certain of matching anything we'd have to read the whole file in. I'm sure there must be some simple way around this problem as many people use searchable flat files with PHP.

Cheers
Bubble

drawmack

backtrack one byte for each charater in the phrase and prepend that to the begining of the next read.