Headless browsers for scraping

[NathanRidley]

Hey, thought I'd point you guys at an answer I posted at StackOverflow a while back, containing a list of headless browsers for various different languages. Might help you if you're getting started with scraping.

automation - headless internet browser? - Stack Overflow

 

[tainted]

Thanks Nathan. If yall want a headless browser with strong JS support, I believe Awesomium is the most robust one in that list. Consider checking it out

-t

 

[NathanRidley]

Yep, that's my choice for my development. If you make more than $100k/year you have to pay though, according to the license terms.

 

[dchuk]

the new cool kid on the block is phantom.js, with from my limited experience with it has been pretty awesome.

 

[NathanRidley]

Phantom is cool, though has its own issues. I recall having problems with font loading and some other miscellaneous issues that I don't recall. Awesomium is the most robust I think, but lower level; you'll have to write some framework code around it in order to get it automating, and that'll either be in .Net or C++, which limits the target market again.