Hi there,
I have a question about cURL, I'm not sure if what I'm trying to accomplish is possible.
Let's say I have 200 URL's that I want to fetch using Curl_multi functions. I've seen plenty of functions to do that, but they all send your entire list of URL's to cURL at once and tell cURL to process them all at the same time, which I imagine must put a lot of stress on the server.
What I want to do is set an amount of "threads" for cURL, so I start by sending the first 10 URL's to cURL, and as soon as one page is done downloading, I send the next URL on the list to cURL, until the list is exhausted. That way I only have a maximum of 10 downloads running at any given time, while still benefiting from multithreaded cURL.
Is this possible?
If it is, does anyone have any pointers or code that might help me code this?
Thanks in advance
