merging two arrays - more integrally than array_merge()

[sfullman]

I'd like to write a function which merges a second array "onto" a first array, so that only addition of nodes takes place unless the first array "ending" node is different. Look at the two arrays and then the resulting array I want, and you'll see what I mean. array_merge() does NOT work for this:

//this is the first array
$a1=array(
	'key1'=>array(
		'cats'=>array(
			1=>'hello there',
			2=>'18 signals',
			3=>'inet buffer',
			'podcast'=>'1 hour long'
		)
	),
	'key2'=>array(
		'cosmo'=>5,
		'questions'=>'antidisestablishmentarianism',
		5=>'cosmo',
		'yec'=>array(
			'three'=>1,
			'blind'=>2
		)
	),
	7=>'elegant'
);
//this is the second array which is merged into or onto the first
$a2=array(
	'key1'=>array(
		'cats'=>'overwritten!',
		'dogs'=>array(
			'rule'=>'over cats'
		)
	),
	'key2'=>array(
		'questions'=>'short'
	),
	'key3'=>array(
		'not'=>'here',
		'but'=>'added'
	)
);

//this is the result of the merge
$result=array(
	'key1'=>array(
		'cats'=>'overwritten!',
		'dogs'=>array(
			'rule'=>'over cats'
		)
	),
	'key2'=>array(
		'cosmo'=>5,
		'questions'=>'short',
		5=>'cosmo',
		'yec'=>array(
			'three'=>1,
			'blind'=>2,
			'mice'=>3,
			'see how they run'=>4
		)
	),
	'key3'=>array(
		'not'=>'here',
		'but'=>'added'
	),
	7=>'elegant'
);

As I think about this I realize I need a pretty elegant algorithm, yet comprehensibly it's just superposing one tree on another which will either add a branch or overwrite a branch. anyone written a function like this?

Thanks

[bpat1434]

Have you looked at [man]array_merge_recursive/man? Seems to do what you want except maybe the numerical indexed items.

[sfullman]

Hi bpat 1434 - NO array_merge_recursive() has the problem that if the two arrays share a node and one is an array and the other is a string, the string gets appended to the array in the other vs. having the comparison array's string node overwrite the reference array's "array" node.

The purpose of this function is to have an array representing, say, settings. Then declare separately another "custom user settings" array. If the 2nd array has any differences, they will overwrite the first array.

Additionally, array_merge_recursive() re-indexes keys in some cases. try running it on $a1 and $a2 below and you'll see.

I COULD STILL USE HELP ON THIS, NOTE THE LINE

if($str)

- I had to add that because on recursing, the whole array got overwritten. Uncomment the echo lines and you'll see what i mean.

My solution isn't "pretty" but it gives the results that a human being would understand and predict, I have included the code below:

<pre>
<?php

//this is the first array
$a1=array(
	'key1'=>array(
		'cats'=>array(
			1=>'hello there',
			2=>'18 signals',
			3=>'inet buffer',
			'podcast'=>'1 hour long'
		)
	),
	'key2'=>array(
		'cosmo'=>5,
		'questions'=>'antidisestablishmentarianism',
		5=>'cosmo',
		'yec'=>array(
			'three'=>1,
			'blind'=>2
		)
	),
	7=>'elegant'
);
//this is the second array which is merged into or onto the first
$a2=array(
	'key3'=>array(
		'not'=>'here',
		'but'=>'added'
	),
	'key1'=>array(
		'cats'=>'overwritten!',
		'dogs'=>array(
			'rule'=>'over cats'
		)
	),
	'key2'=>array(
		'questions'=>'short'
	)
);

//these 2 vars are just for testing
$passcounter1=$passcounter2=0;

function array_merge_accurate(){
	$arrays=func_get_args();
	if(count($arrays)==0)return '';
	if(count($arrays)==1)return $arrays[0];
	//store the first array passed
	$firstArrayIdentifier='a'.md5(time().rand(1,100000));
	global $$firstArrayIdentifier;
	$$firstArrayIdentifier=$arrays[0];
	for($i=1; $i<=count($arrays); $i++){
		array_merge_accurate_subroutine($firstArrayIdentifier, $arrays[$i]);
	}
	return $$firstArrayIdentifier;
}
function array_merge_accurate_subroutine($firstArrayIdentifier,$comparisonArray, $keys=array()){
	global $passcounter1, $passcounter2, $$firstArrayIdentifier;
	//where are we
	if($keys){
		$str='';
		foreach($keys as $build)$str.='['.(is_numeric($build)?'':'\'').$build.(is_numeric($build)?'':'\'').']';
	}
	#echo( '<span style=color:darkred>$mylocation=$comparisonArray'.$str.';</span><br>' );
	eval( '$mylocation=$comparisonArray'.$str.';' );
	if(is_array($mylocation)){
		$passcounter1++;
		foreach($mylocation as $n=>$v){
			#echo 'calling again '.$passcounter1.':' . $passcounter2.' with key <strong>'.$n.'</strong> and keys <strong>'.(implode('.',$keys) ? implode('.',$keys) : '-').'</strong><br />';
			array_merge_accurate_subroutine($firstArrayIdentifier, $comparisonArray, array_merge($keys,array($n)));
		}	
	}else{
		$passcounter2++;
		#echo('pass '.$passcounter1.':'.$passcounter2.' $'.$firstArrayIdentifier.$str.'='.(is_numeric($mylocation)?'':'"').str_replace('"','\"',$mylocation).(is_numeric($mylocation)?'':'"').';<br />');
		if($str)
		eval('$'.$firstArrayIdentifier.$str.'='.(is_numeric($mylocation)?'':'"').str_replace('"','\"',$mylocation).(is_numeric($mylocation)?'':'"').';');
	}
}
$result=array_merge_accurate($a1,$a2);
print_r($result);
//this is the result of the merge
/*
print_r($result);
$result=array(
	'key1'=>array(
		'cats'=>'overwritten!',
		'dogs'=>array(
			'rule'=>'over cats'
		)
	),
	'key2'=>array(
		'cosmo'=>5,
		'questions'=>'short',
		5=>'cosmo',
		'yec'=>array(
			'three'=>1,
			'blind'=>2,
			'mice'=>3,
			'see how they run'=>4
		)
	),
	'key3'=>array(
		'not'=>'here',
		'but'=>'added'
	),
	7=>'elegant'
);
*/
?>

[paulnaj]

Hi,

Your 'result' array contains ...

       'yec'=>array(
            'three'=>1,
            'blind'=>2,
            'mice'=>3,
            'see how they run'=>4
        ) 

... but I don't see where the last two entries come from ('mice'=>3 and 'see how they run'=>4).

If that's just a cut'n'paste mistake, then you could try the following function (for two arrays only - easy to expand to multiple arrays) ...

function array_merge_recursive_test($a1, $a2){
	$a1_keys = array_keys($a1);
	foreach($a2 as $key2 => $val2){
		if(in_array($key2, $a1_keys) && is_array($val2)){
			$val = array_merge_recursive_test($a1[$key2], $val2);
		} else {
			$val = $val2;
		}
		$a1[$key2] = $val;
	}
	return $a1;
}

$a = array_merge_recursive_test($a1, $a2);
echo '<pre>'; print_r($a); echo '</pre>';

P.

[bpat1434]

I think your mini-function falls short in that sfullman wants to overwrite a key with an array if the original is a string, and vice-versa. So there's a bit more logic to be done.

Really I wonder what's making the two arrays, and why they need to be merged. Something this complicated just screams bad design or a flaw in logic somewhere else.

[paulnaj]

Hi bpat,

sfullman wants to overwrite a key with an array if the original is a string, and vice-versa.

Not sure what you mean by vice-versa here. All keys from the first array are overwritten by duplicate keys from the second array.

But I did miss one possiblility. The cases for duplicate keys are:

1. $val1 is array and $val2 is array => merge recursively
2. $val1 is not array and $val2 is array => $val2 overwrites $val1
3. $val1 is array and $val2 is not array => $val2 overwrites $val1
4. $val1 is not array and $val2 is not array => $val2 overwrites $val1

($val1 appears as $a1[$key2] in the function)

I missed #2 - didn't crop up in the example! So, the function should be...

function array_merge_recursive_test($a1, $a2){
    $a1_keys = array_keys($a1);
    foreach($a2 as $key2 => $val2){
        if(in_array($key2, $a1_keys) && is_array($a1[$key2]) && is_array($val2)){
            $val = array_merge_recursive_test($a1[$key2], $val2);
        } else {
            $val = $val2;
        }
        $a1[$key2] = $val;
    }
    return $a1;
}

P

[bpat1434]

Your scenario for #1 is incorrect. If you read the initial post from sfullman he states that if the two values are arrays then the second arrays values should be added...

sfullman wrote:

only addition of nodes takes place unless the first array "ending" node is different.

So if val1 is an array, and val2 is an array, then there will be no "merging" but only addition. Unless the last element in val1 is different from something in val2. I'm not sure what in val2, the entire thing or just the last node in val2, or the first node.

[paulnaj]

1. $val1 is array and $val2 is array => merge recursively

I should have been clearer... merge recursively here means using the function array_merge_recursive_test() recursively, not the in-built array_merge() or any other "merge".

I don't think I understand what 'ending' node is here - surely it's not 'last' one (that would be listed in array_keys()), since that would have unexpected results.

You don't seem to know either so I'm surprised you can say categorically that my function is incorrect. It certainly doesn't fail the example given.

[bpat1434]

My point was that if they are each arrays, they shouldn't be merged; rather, only keys that aren't in array1 that are in array2 should be added. Unless the last element in array1 isn't the same as something in array2. What that something is, I'm not sure. In that instance, then the entire value of the current key in array1 is replaced by the entire value of the same key in array2.

Sfullman's post didn't do a whole lot to show all examples and fringe scenarios that he wants, so I can only estimate that your function, while more in line with what I would do, still isn't 100% right since it doesn't cover all the bases of sfullman's original post.

Here's code I've come up with that does what he says EXCEPT for where the whole mice => 3, see how they run => 4 comes into play. I don't see that in his examples, so I'm leaving it out. Here's my code:

<?php

ini_set('display_errors', 1);

$array1 = array
(
    'key1' => array
    (
        'cats' => array
        (
            1 => 'hello there',
            2 => '18 signals',
            3 => 'inet buffer',
            'podcast' => '1 hour long'
        )
    ),
    'key2' => array
    (
        'cosmo' => 5,
        'questions' => 'antidisestablishmentarianism',
        5 => 'cosmo',
        'yec' => array
        (
            'three' => 1,
            'blind' => 2
        )
    ),
    7 => 'elegant'
);

$array2 = array
(
    'key1' => array
    (
        'cats' => 'overwritten!',
        'dogs' => array
        (
            'rule' => 'over cats'
        )
    ),
    'key2' => array
    (
        'questions' => 'short',
        /*
        'yec' => array
        (
            'mice' => 3,
            'see how they run' => 4
        )
        */
    ),
    'key3' => array
    (
        'not' => 'here',
        'but' => 'added'
    )
);

function sfullman_array_merge_recursive()
{
    $arrays = func_get_args();
    if(!count($arrays))
    {
        return null;
    }
    elseif(count($arrays) == 1)
    {
        return $arrays[0];
    }

// Store the first array and merge all others onto it
$array1 = array_shift($arrays);

for($i=0; $i<count($arrays); $i++)
{
    $array2 = $arrays[$i];
    foreach($array2 as $key => $value)
    {
        // If it's new to array1, add it and continue
        if(!isset($array1[$key]))
        {
            $array1[$key] = $value;
            continue;
        }

        if(is_array($array1[$key]))
        {
            $lastItem = end($array1[$key]);

            if(is_array($lastItem))
            {
                // Merge the array recursively, following the same rules again...
                $array1[$key] = sfullman_array_merge_recursive($array1[$key], $value);
                continue;
            }

            // If they're not the same, replace what's in array1 with what's in array2
            if($lastItem != $value)
            {
                $array1[$key] = $value;
            }
        }

        // If we're here, the keys are duplicates, so overwrite array1 with array2 value
        $array1[$key] = $value;
    }
}

return $array1;
}

$var = sfullman_array_merge_recursive($array1, $array2);

?>
<pre><?php echo print_r($var, 1); ?></pre>

That produces this array:

Array
(
    [key1] => Array
        (
            [cats] => overwritten!
            [dogs] => Array
                (
                    [rule] => over cats
                )

    )

[key2] => Array
    (
        [cosmo] => 5
        [questions] => short
        [5] => cosmo
        [yec] => Array
            (
                [three] => 1
                [blind] => 2
            )

    )

[7] => elegant
[key3] => Array
    (
        [not] => here
        [but] => added
    )

)

[paulnaj]

It'd be interesting to know what sfullman thinks.

[sfullman]

It appears that both functions work OK, but in plain english what I'm doing is superposing one "tree" on another, node by node.

If a node is an array, and the superposed node is a string, it "cuts off" all data in the subject node, but as long as the branches match up, it will go as deep as the initial array.

Obviously, there is no "replacing", only addition. There's no deletion either, though I could set a node's value to null but that's not truly "unsetting" the node.

As to how it would be used. If I know intimately the structure of an array for settings, I can write a compact array that mirrors the settings array, changing values I need to, even changing a string value to an array if that's a meaningful option, and adding options on the same level as other options (nodes).

Samuel

[Weedpacket]

sfullman wrote:

plain english

...isn't very good for specifying algorithms 🙂

If a node is an array, and the superposed node is a string, it "cuts off" all data in the subject node, but as long as the branches match up, it will go as deep as the initial array.

Obviously, there is no "replacing", only addition. There's no deletion either, though I could set a node's value to null but that's not truly "unsetting" the node.

Not sure what you mean when there's no "replacing", since the array that was originally in $array1['cats'] was replaced by a string (but I guess that's what was meant by "cuts off"). If I do understand you right I interpret what you want as:

Given two arrays $default and $custom:
	For each key $k in $custom
		if $default[$k] does not exist, then set $default[$k] = $custom[$k]
		if either $default[$k] or $custom[$k] is not an array, then set $default[$k] = $custom[$k]
		Otherwise, set $default[$k] to the result of running this algorithm on $default[$k] and $custom[$k]
	return $default

That translates into PHP as

function override_defaults($default, $custom)
{
	foreach($custom as $k=>$v)
	{
		if(!array_key_exists($default, $k))
		{
			$default[$k] = $custom[$k];
		}
		elseif(!is_array($default[$k]) || !is_array($custom[$k]))
		{
			$default[$k] = $custom[$k];
		}
		else
		{
			$default[$k] = override_defaults($default[$k], $custom[$k]);
		}
	}
	return $default;
}

Which is substantively the same as paulnaj's code.

I wonder if storing this as a DOM structure would make things easier, given the navigation and manipulation tools it provides, not to mention the free standardised serialisation/deserialisation mechanism. Just a thought.

[sfullman]

try this out and you'll see it doesn't work as expected (desired):

function override_defaults($default, $custom)
{
	foreach($custom as $k=>$v)
	{
		if(@!array_key_exists($default, $k))
		{
			$default[$k] = $custom[$k];
		}
		elseif(!is_array($default[$k]) || !is_array($custom[$k]))
		{
			$default[$k] = $custom[$k];
		}
		else
		{
			$default[$k] = override_defaults($default[$k], $custom[$k]);
		}
	}
	return $default;
}
$a1=array(
    'key1'=>array(
        'cats'=>array(
            1=>'hello there',
            2=>'18 signals',
            3=>'inet buffer',
            'podcast'=>array(
				1=>5,
				2=>6,
				3=>7,
				4=>array(
					'alpha'=>'beta',
					'gamma'=>'delta',
					5,
					6,7
				)
			)
        )
    ),
    'key2'=>array(
        'cosmo'=>5,
        'questions'=>'antidisestablishmentarianism',
        5=>'cosmo',
        'yec'=>array(
            'three'=>1,
            'blind'=>2
        )
    ),
    7=>'elegant'
);
//this is the second array which is merged into or onto the first
$a2=array(
    'key3'=>array(
        'not'=>'here',
        'but'=>'added'
    ),
    'key1'=>array(
        'cats'=>array(
			4=>'inet buffer indeed',
			'podcast'=>array(
				2=>'6b',
				4=>array(
					'gamma'=>array(1,2,3,4,5)
				)
			)
		),
        'dogs'=>array(
            'rule'=>'over cats'
        )
    ),
    'key2'=>array(
        'questions'=>'short'
    )
); 
echo '<pre>';
print_r(override_defaults($a1,$a2));

I added more depth in "podcast" - notice that keys 1 and 3 are not present at all. However, everything else does seem to work.

[paulnaj]

Hi Samuel,

What is the desired result for the last example?

Try and modify weedpacket's pseudo-code...

Given two arrays $default and $custom:
	For each key $k in $custom
		if $default[$k] does not exist, then set $default[$k] = $custom[$k]
		if either $default[$k] or $custom[$k] is not an array, then set $default[$k] = $custom[$k]
		Otherwise, set $default[$k] to the result of running this algorithm on $default[$k] and $custom[$k]
	return $default

... to clarify where it goes wrong.

Paul.

[johanafm]

if(@!array_key_exists($default, $k))

When you need to do this, i.e. add @, you're really doing something wrong.

if (array_key_exists($k, $default))

And then it works like it should.

[Weedpacket]

I had two copies of the code (one for testing, one in the post); fixed the typo in the test code when I got an error message about it, but didn't update the posted code to match.

Cut-and-paste code never works.

[paulnaj]

I've been cutting'n'pasting the wrong bit for years. I'll never learn.

P.

[johanafm]

But at least it's quick 😉

[Weedpacket]

paulnaj wrote:

I've been cutting'n'pasting the wrong bit for years. I'll never learn.

That's what she said.