Good suggestions so far, though i highly recommend you check out phantomjs.org. Phantom is a headless version of WebKit which is the rendering engine behind Chrome & Safari. It's the most comprehensive solution to handling AJAX content when scraping in my book since it's technically the same as interacting with a page loaded by your browser.
-- Dave Kuhn Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Saturday, October 6, 2012 at 3:04 PM, rektide wrote: > Only just picked it up last week, but it worked well enough-- node.io. It > exposes a > jQuery-esque interface for querying scraped pages. Extremely high level, > "just works" > scraping module, in my book! > > It also has a fairly sizable task-processing system built in, which I have > not used. > > Good luck: > https://github.com/chriso/node.io > > -rektide > > On Sat, Oct 06, 2012 at 01:34:03PM -0700, Narek Musakhanyan wrote: > > Hey guys . I tried to scrape a data from a website using PHP cURL lib but > > I failed since cURl allows you to scrape only static content . But the > > content I want to scrape changes via javascript(AJAX) since cURL cant > > hanfle that I couldnt handle scraping via cURL . So I heard the this type > > of things can be done via node . Basically I need to make my node app > > handle this js wait for some time until AJAX is done and the pass it to > > php .So is it possible to do via node.js ? I dont know node and I have to > > start from scratch so I am here you to point out the right node framework > > to use to get the result I explained . > > > > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
