Determining IP's country of origin without 3rd party site?

schwim2

Hey there everyone!

I'm running PHP ver. 5.5.32

I found this function that determines all the good stuff about an IP: http://stackoverflow.com/questions/12553160/getting-visitors-country-from-their-ip

Which I am sorely in need of, specifically the country of origin, but I'm wondering if there's a way to do this without tapping into a remote web resource. Can this be done with PHP's GeoIP? I've been reading through it but I just can't figure out how to implement it, if it in fact can do what I need.

Thanks for your time!

NogDog

I might look at downloading Maxmind's GeoLite2 Country database, which they update monthly -- and which is free. http://dev.maxmind.com/geoip/geoip2/geolite2/

You could pull down the CSV version, then load it into whatever DBMS you are using, and presumably just write some queries against it to get the country for the IP in question.

NogDog

Or download their binary DB and use their PHP API with it. 🙂

sneakyimp

This stuff sounds interesting. I would point out that nothing is stopping your visitors from using a VPN or just firing up some remote machine on Amazon EC2 or whatever and using that to access your site.

schwim2

Hi there Sneaky,

You're absolutely right but some don't, so I write stuff to handle those. If I ignored exploit detection that could be circumvented, I'd have no exploit detection at all 😃

There's a particular issue that I was trying to solve with this:

To be brief, my function holds unique IPs for 30 days of inactivity. If it's never been seen before, it does the lookup and if it has been seen before, it just updates the lastseen time. In the image above, you can see that the site is at times getting hammered from a particular region and all the page loads are malicious in nature. To save going through all the URL and UA checks, this function allows me to quietly blackhole them, save resources and not have to even see them in the ban panel, so I can still keep an eye on new and different exploit attempts. It's done a great job at cleaning up my view on the page.

Since I'm not smart enough to figure out exploits before I see them, I often just write these things to handle very particular cases that I come across. I think I started a thread here concerning Yandex redirect URLs but in that case, I was getting URLs attempting to access the admin panel all with a Yandex redirect URL as the referer so I wrote a function to ban all traffic coming with a referer from that domain. It works well for me because this particular site sells auto parts in the US only and I don't mind losing any and ALL RU traffic, much less that from the Yandex search engine. Since I implemented it, it's captured thousands of these malicious page loads.

sneakyimp

If you are experiencing malicious behavior (aren't we all?) that is routed via/filtered through/coming from Yandex, I might suggest that you ban their entire IP block using iptables or something. This would be a fast-and effective way to ignore vast ip blocks entirely that would not tax your web app at all. E.g., if you do a network lookup for 202.46.58.136, you can see that there's a big block allocated to ShenZhen Sunrise Technology:

inetnum:        202.46.32.0 - 202.46.63.255
netname:        SUNRISE
descr:          ShenZhen Sunrise Technology Co.,Ltd.
descr:          2002 Jiabin Road,Luohu District,ShenZhen,China
country:        CN
admin-c:        MM546-AP
tech-c:         MM546-AP
mnt-by:         MAINT-CNNIC-AP
mnt-irt:        IRT-CNNIC-CN
mnt-routes:     MAINT-CNNIC-AP
status:         ALLOCATED PORTABLE
changed:        hm-changed@apnic.net 20050705
changed:        hm-changed@apnic.net 20151202
source:         APNIC

Seems to me you could formulate an iptable rule to drop all requests from this company fairly easily. Using a CIDR utility you an enter the starting ip range of 202.46.32.0 and an ending one of 202.46.63.255 and the tool will tell you that corresponds to CIDR 202.46.32.0/19. You can add a rule to drop all requests from this company using this command:

sudo iptables -I INPUT 30 -s 202.46.32.0/19 -j DROP

I realize that you cannot use this approach to actively detect new, fresh bad guys, but it's almost perfectly effective and would not tax your server much at all.

If you need to detect malicious behavior, it's helpful if ALL of your page requests are routed through a single PHP script -- this is how some frameworks (like CodeIgniter) work. You have some rewrite rule to map SEO-friendly urls onto this one file (e.g., index.php) but with query string params to select the correct functionality. If you have this, you can easily install any sort of every-page processing you need. You can have a function to scour user agents or a function to block IP addresses or some kind of heuristic sniffer to detect malicious activity and then apply some ban method -- although I must warn you that a huge IP block like 202.46.32.0/19 might result in a LOT of ip addresses ending up in a ban_table somewhere which can be quite inefficient. You might consider sending a 403 GONE result or something. If the attack is automated this is sort of like 'playing 'possum'

I had an issue where I was getting millions of SQL injection attempts from bad guys the world over -- there was no order or rhyme or reason to the IP addresses from which this attack came so I presume they were using a botnet. I added a filter that used a regex to sniff for any of the guilty sql injection hacks by looking for SELECT.*UNION or CONCAT or various other patterns that would never be sent by a legitimate request. I hesitate to say that this has truly solved the problem, but it shut the bad guys down pretty fast and got my server back to nominal functioning.

I've run into apache modules that do this kind of thing in a pretty serious way. I believe it was ModSecurity and it had certain problems sometimes -- e.g., it might interfere with actual proper site functions and it was difficult to track down the source of certain problems until I realized that apache was using pattern matching to auto-send 4xx responses when the request url matched some suspicious-looking pattern.

schwim2

sneakyimp;11059449 wrote:
If you need to detect malicious behavior, it's helpful if ALL of your page requests are routed through a single PHP script -- this is how some frameworks (like CodeIgniter) work. You have some rewrite rule to map SEO-friendly urls onto this one file (e.g., index.php) but with query string params to select the correct functionality. If you have this, you can easily install any sort of every-page processing you need. You can have a function to scour user agents or a function to block IP addresses or some kind of heuristic sniffer to detect malicious activity and then apply some ban method -- although I must warn you that a huge IP block like 202.46.32.0/19 might result in a LOT of ip addresses ending up in a ban_table somewhere which can be quite inefficient. You might consider sending a 403 GONE result or something. If the attack is automated this is sort of like 'playing 'possum'

I had an issue where I was getting millions of SQL injection attempts from bad guys the world over -- there was no order or rhyme or reason to the IP addresses from which this attack came so I presume they were using a botnet. I added a filter that used a regex to sniff for any of the guilty sql injection hacks by looking for SELECT.*UNION or CONCAT or various other patterns that would never be sent by a legitimate request. I hesitate to say that this has truly solved the problem, but it shut the bad guys down pretty fast and got my server back to nominal functioning.

This is exactly how I handle it, sneaky. One php file in the publicly accessible directory and htaccess routes all traffic through it. That's what's allowed me to get an idea of what they try even when it falls on a 404. I have also written the system to look for SQL stuff and I can automatically ban based on URL string, file, query match, IP(and anything associated with it), user agent, referer and other aspects . I've just begun playing with heuristic type detection, looking to label the wget, zagrab, curl requests that flood the site but have an altered user agent, so they don't get caught in the normal manner.

This aspect of building a site is actually my most enjoyed. It would be different, I'm sure, if 99% of malicious requests weren't performed by the low-hanging fruit of the bunch and so easily detected. It would probably be a nightmare if more than a minuscule percentage of these guys took any pride or put any effort in their attempts.

sneakyimp

schwim2;11059459 wrote:
I've just begun playing with heuristic type detection, looking to label the wget, zagrab, curl requests that flood the site but have an altered user agent, so they don't get caught in the normal manner.

This sounds pretty interesting. Could you elaborate what you mean by 'heuristic type detection' ?

schwim2;11059459 wrote:
This aspect of building a site is actually my most enjoyed. It would be different, I'm sure, if 99% of malicious requests weren't performed by the low-hanging fruit of the bunch and so easily detected. It would probably be a nightmare if more than a minuscule percentage of these guys took any pride or put any effort in their attempts.

I agree that security considerations are a good bit more interesting than just building a site -- the adversarial aspect is way more interesting than centering one more DIV tag or improving performance on a db structure.