[RESOLVED] Website Filtering

[ryneaux]

Hi all!

I am working on a sort of link directory and I want to filter out submission of sites which have a suggestive and/or restrictive rule for those who are under the age of 18.

I have been looking around for some sort of accumulated list of such sites in order to give the filtering of sites a decent start, because after the "start" entries will have to be viewed manually in order to know whether or not the site's content is acceptable.

Anyway, if any one knows where I might find such a list I would appreciate it.

Thanks...

[sneakyimp]

There is probably no such thing as a list that will block all of them, but I use the famous HOSTS file linked from this site. You'd have to parse it, but it might be a good place to start.

You might also be able to crawl a site like this one or this one and extract a list of domains to ban using some pattern matching.

[ryneaux]

Cool thanks, I have found a couple of other lists in the meantime as well and have a list of about 30000 sites. A pretty good start.

thanks again!