[RESOLVED] Extract string from a file

[JimCook]

I am trying to extract a list of IP addresses from a log file with the following google code and not having much luck

<?
$file = file_get_contents('http://www.mysite.com/log.html');
preg_match_all('/^([1-9]¦[1-9][0-9]¦1[0-9][0-9]¦2[0-4][0-9]¦25[0-5])(\.([0-9]¦[1-9][0-9]¦1[0-9][0-9]¦2[0-4][0-9]¦25[0-5])){3}$/',$file,$a);

$count = count($a[1]);
echo "<b>Number of hack attempts ips</b> = " .$count."<p>";
for ($row = 0; $row < $count ; $row++) {
echo $a[1]["$row"]."<br>";
}

?>

The log file looks like

Referer: http://www.site.com/page1.htm, Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.1), 203.26.142.253, Sunday 21st of December 2008 07:13:46 PM

Any help on this would be great I am trying to get a list of IPs in this format

127.0.0.1
10.10.10.1
192.168.0.1
ect

[bradgrafelman]

Try something like:

preg_match_all('/\s([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}),\s/', $file, $a);

[NogDog]

It looks like you may be using the wrong "or" character.

echo "¦";

...should look like...

echo "|";

...in this forum's PHP tags.

[JimCook]

I think I had 2 problems as I have no error checking in the file handle the file was not getting picked up correctly. The pipe command was also wrong :-) Just checking the file now and all seems to be Ok will close the topic in a second if it works.

[nrg_alpha]

Here's my take:

$str = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; 197.26.42.2; .NET CLR 2.0.50727; InfoPath.1), 203.26.142.253, Sunday 21st of December 2008 07:13:46 PM'; // just some string with a few random IP addresses in it...
preg_match_all('#((?:[0-9]{1,3}\.){3}[0-9]{1,3})#', $str, $matches);
echo '<pre>'.print_r($matches[1], true);

Output:

Array
(
    [0] => 197.26.42.2
    [1] => 203.26.142.253
)

EDIT - Come to think of it, I can lose the capture parenthesis and just output $matches[0] instead...

preg_match_all('#(?:[0-9]{1,3}\.){3}[0-9]{1,3}#', $str, $matches);

[NogDog]

You can save a little typing with the '\d' escape sequence:

'/(\d{1,3}\.){3}\d{1,3}/'

[nrg_alpha]

This is true.. but ever since Weedpacket showed me that \d matches more than simply 0-9, I got into a habit of strictly specifying [0-9] ...

Alos note that in your sample, there is no need to have any captures, so the parenthesis should start with ?:

[NogDog]

nrg_alpha;10896658 wrote:

This is true.. but ever since Weedpacket showed me that \d matches more than simply 0-9,...

I'll need to see a reference or proof before I'll worry about that, since (a) the manual says that \d matches any decimal digit and I'm only aware of 10 such digits, plus (b) it's never yet failed me. (Possibly there could be issues if scanning multi-byte unicode text, perhaps?)

[nrg_alpha]

NogDog;10896661 wrote:

I'll need to see a reference or proof before I'll worry about that, since (a) the manual says that \d matches any decimal digit and I'm only aware of 10 such digits, plus (b) it's never yet failed me. (Possibly there could be issues if scanning multi-byte unicode text, perhaps?)

Ask and thou shalt receive 😉

$str = 'this is text 123²4³5 this is more text...';
preg_match_all('#\d+#', $str, $matches);
echo '<pre>'.print_r($matches[0], true);

Output:

Array
(
    [0] => 123²4³5
)

Note the exponents being matched as well..

Now granted, in this situation that the OP is looking for, I would doubt an encounter with exponents.. so yes, in general, \d is deemed safe enough. But one of these days, someone will use \d when it WILL match more than merely 0-9, and they will scratch thier heads wondering why.. so just as a force of habit, (more often than not.. but not all the time), I tend to explicitly use [0-9].

I have not actually timed the speed difference between the two methods in matching say the above text in a 10,000 time repeating loop.. but I wouldn't be surprised if [0-9] edges out \d, just for that simple reason (that \d as a character class checks for slightly more characters...

EDIT - ..Oops.. the above string would not make a good comparison.. stript out all the exponents and then compare [0-9] vs \d would be more fair 😉

[JimCook]

Thanks guys I love it when we get down and dirty with the code.

I did try and use this expression but it seems to split it on the first octets and the output was all over the place.

/(\d{1,3}.){3}\d{1,3}/

(I used it in the original code)

[nrg_alpha]

JimCook;10896663 wrote:

Thanks guys I love it when we get down and dirty with the code.

I did try and use this expression but it seems to split it on the first octets and the output was all over the place.

(I used it in the original code)

Could you show a sample of what you are plugging into the regex , and what it is spitting out?

[nrg_alpha]

Jim, it just donned on me.. when you are using

/(\d{1,3}\.){3}\d{1,3}/

and I'm going to assume for sake of argument that you are using $matches as your array that stores the results, are you using $matches[0] or $matches[1]?
In the case of the snippet above, you will want to use $matches[0], as that sample has an un-necessary set of capturing parenthesis... so if you are outputing $matches[1], it would explain the incorrect results you are looking for, as $matches[1] will output \d{1,3}. three times..but not the whole pattern.

[NogDog]

nrg_alpha;10896662 wrote:

Ask and thou shalt receive 😉 ...

Curious. Now the question is: is it a bug or a feature?

[nrg_alpha]

NogDog;10896667 wrote:

Curious. Now the question is: is it a bug or a feature?

Well, as per your definition, it is correct in that it matches 'digits' (I suppose one could still consider exponents as mini-digits?). Whether it is a bug or not, I can't say for sure.

[JimCook]

nrg_alpha;10896665 wrote:

Jim, it just donned on me.. when you are using

/(\d{1,3}\.){3}\d{1,3}/

and I'm going to assume for sake of argument that you are using $matches as your array that stores the results, are you using $matches[0] or $matches[1]?
In the case of the snippet above, you will want to use $matches[0]

Here is the code that I used with the shorter preg expression that did not work as expected. 😕

<?
$file = file_get_contents('log.html');
preg_match_all('/(\d{1,3}\.){3}\d{1,3}/', $file, $a);
$count = count($a[0]);
echo "<b>Number of ips</b> = " .$count."<p>";
for ($row = 0; $row < $count ; $row++) {
echo $a[1]["$row"]."<br>";
}

?> 

And the output :eek:

Number of ips = 108

0.
245.
124.
0.
44.
0.
44.
155.
190.
11.
163.
230.
0.
201.
72.
0.
76.
118.
113.
142.
0.
117.
11.

This is the code that I was able to use it appears to output the code pretty well.
I tired it on a log file that was about 2mb and it outputted about 5k ips in about 40 seconds

🙂

<?
$file = file_get_contents('log.html');
preg_match_all('/\s([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}),\s/', $file, $a);
$count = count($a[1]);
echo "<b>Number of ips</b> = " .$count."<p>";
for ($row = 0; $row < $count ; $row++) {
echo $a[1]["$row"]."<br>";
}

?> 

Number of ips = 77

122.104.245.32
114.77.124.208
210.49.44.17
210.49.44.17
114.129.155.85
121.223.190.67
125.255.11.66
124.185.163.145
203.25.230.151
220.233.201.36

[nrg_alpha]

JimCook;10896671 wrote:

Here is the code that I used with the shorter preg expression that did not work as expected. 😕

<?
$file = file_get_contents('log.html');
preg_match_all('/(\d{1,3}\.){3}\d{1,3}/', $file, $a);
$count = count($a[0]);
echo "<b>Number of ips</b> = " .$count."<p>";
for ($row = 0; $row < $count ; $row++) {
echo $a[1]["$row"]."<br>";
}
[/QUOTE]

What catches my eye immediately is that you are echoing out $a[1]["$row"]. I think you should be working off of $a[0], not $a[1]... If you are just trying to list everything stored within $a in the preg statement.. perhaps this will work out as well? (untested)

[code=php]
$file = file_get_contents('log.html');
preg_match_all('/(?:\d{1,3}\.){3}\d{1,3}/', $file, $a);
for($i = 0, $total = count($a[0]); $i < $total; $i++){
   echo $a[0][$i] . '<br />';
}

[NogDog]

In the first case, the entire pattern match is in $a[0], not $a[1]:

echo $a[0]["$row"]."<br>";
//      ^

[JimCook]

I know we fixed the code but its sort of fun working on this

The last example gave this out put

[/CODE]
1.9.0.4
122.104.245.32
114.77.124.208
1.9.0.5
210.49.44.17
1.9.0.5
[/CODE]

[bradgrafelman]

Note you could also replace:

for($i = 0, $total = count($a[0]); $i < $total; $i++){
   echo $a[0][$i] . '<br />';
} 

with:

echo implode('<br />', $a[0]);

[JimCook]

Thanks bradgrafelman and for every ones posts we are taking the ip address out of the log files working out the IP address range and then adding them to a blocked range on our webserver.

At the moment we have ranges like

119.59.80.0 - 119.59.87.255
121.58.160.0 - 121.58.167.255
121.100.48.0 - 121.100.55.255

And need to turn them in to some thing like

If REMOTE_ADDR !^{(119.59.80.0:119.59.87.255)}

If REMOTE_ADDR !^{(121.58.160.0:121.58.167.255)}

If REMOTE_ADDR !^{(121.100.48.0:121.100.55.255)}

I am using excel with search and replace as you can guess this is taking a very long long time to do any way in php this can be done ?