Match names to Wikipedia records

zzz

I need to run through a large list of names and see if any of these people are listed on Wikipedia (some of them are).

Any idea where I start?

Kudose

Depending on the list size, I would ask Wikipedia permission to hit the server that many times and ask an acceptable hit rate.

zzz

But how do I do it anyway?

I can write a loop and load, say 50 records at a time, put results in iFrames and scan through them. That's the best i can think of. Any more ideas?

Kudose

I would use cURL to hit Wikipedia. It has the ability to make concurrent calls.

How I would proceed depends on your end goal.

zzz

OK, I think this is way out of my league. I think I better go a low tech way, loop through my names, append them to Wikipedia's search URL and load results into iframe, big enough to see if there's a match. Do in chunks of 50 names at a time not to trigger flood control. When i find a match I actually have to go manually to each page and add something.

johanafm

If all you want is to know wether the article exists or not, inspecting the response header is enough. Wikipedia sends a 404 Not Found in this case.

zzz

Ah, interesting. I'm willing to give it a whirl. How do I do that?

dagon

get_headers()