I am trying to write a program that collects the links from my favorite TV show from hulu: http://www.hulu.com/outer-limits

The problem is that many episodes are nested under previous seasons that only show up in the html source when you click on the season. Is there a way to make curl "click" on the old seasons to make the episode listings show up in the html?

    pianoparadise;10938313 wrote:

    Is there a way to make curl "click" on the old seasons

    Sort of. The click you refer to here end up in an XHR request, which is nothing but an http request, aside from the fact that the browser doesn't treat it as a new page request and that it's initiated through javascript. For the purpose of curl, forging the same request will yield the same results, initially anyway.

    If you don't allready use Firefox, I recommend you get it, and also get the addon Firebug. Rightclicking on one of the seasons and clicking "inspect element" shows you

    <td id="s-3" colspan="6" class="vex-up" onclick='VideoExpander.subheadingClicked("s-3", "http://www.hulu.com/videos/season_expander?order=desc&amp;page=1&amp;season_number=3&amp;show_id=189&amp;sort=season&amp;video_type=episode")'>Season 3<span style="display: none;" class="srh-loading"><img width="16" height="16" border="0" src="http://static.huluim.com/images/loading-animated-circle.gif?1262441028" alt="Loading-animated-circle"/></span></td>

    Or, you could click Firebug's "Net" menu, and then left click an episode link to see the request sent by the browser.
    Anyway, in the above, "s-3" is the id of the clicked element, and the second argument is the requested URI. So, you don't need to do anything but request that resource, and do whatever you need to with the results.

    But if you do this strictly with php/curl and wish to use DOM or Xpath to get the episode info, some additional work is needed since the response contains character escape sequences, such as \u003c and \u003e for < and > respectively.
    So it may be easier to initiate the request to your host through javascript, use curl to retrieve the information from hulu and pass it back to your javascript which can then be used to translate those characters more or less automatically, so you can process it with DOM, either with javascript or by passing it back to your server for php processing.

    For example, using the (relevant part of the) string returned by the resource for season 5 with this javascript

    function showStr() {
	var d = document;
	var tmp = d.createElement('div');
	tmp.innerHTML = str;				// str is part of the returned string for s-5

var div = d.getElementById('here');
var alist = tmp.getElementsByTagName('a');
for (var i = 0; i < alist.length; ++i) {
	div.appendChild(d.createTextNode(alist[i].innerHTML));
	div.appendChild(d.createElement('br'));
}
}

    produces output like this

    Better Luck Next Time
<img alt="Add this video to your queue" class="hover-me" id="icon-vex-add.gif126256923512572" src="http://static.huluim.com/images/icon-vex-add.gif?1262441028" title="Add this video to your queue" border="0">
Fathers &amp; Sons
<img alt="Add this video to your queue" class="hover-me" id="icon-vex-add.gif126256923512759" src="http://static.huluim.com/images/icon-vex-add.gif?1262441028" title="Add this video to your queue" border="0">
      Write a Reply...