You're ignoring the current/parent dir, but you actually need to use them. This looks to me like a job for some multi-dimensional arrays, or perhaps just one. You'd have to track how deep and high the array gets using counters, but lets say you have a folder 'A' with 3 sub-folders 'A1', 'A2', and 'A3', and there are 3 files in each folder. You want to build an array that looks like this:
$listing['A'] =>
($folder['A1'] => ("file1", "file2", "file3"),
$folder['A2'] => ("file4", "file5", "file6"),
$folder['A3'] => ("file7", "file8", "file9"))
Of course, your listing array will probably go deeper and wider than that, but hopefully you get the picture. The height and depth counters will help you access the array elements later, so use those as you build your code.
I'm also envisioning some functions here. Something like:
if($file == '.')
{
$depth = setCurrent($file);
continue;
}
if($file == '..')
{
setParent($file);
continue;
}
if(isdir($file))
$height = setFolder($file);
else
$height = setFile($file);
Hopefully this will point you in the right direction. Obviously, there's more code involved here, but I'll leave that to you or anyone who has time to think it all out.