Eventlet?

[mattseh]

I've been playing with it, sick of normal threads of scrapers etc. It seems like less green threads gives better performance? Any advice / tips?

 

[dchuk]

90% of the time your threads are speaking to empty ears here buddy

 

[ArgentCloud]

Eventlet probably outperforms many alternatives, but if memory serves you can manage your own async IO with urllib2 and friends so it's probably overkill for things like scrapers.

 

[brentdev]

greenlet vs eventlet - Raymond Tay: Python's Greenlet/Linden Lab's Eventlet

but u may want to check this project out - syncless - Project Hosting on Google Code
builds in a lot of the goodies that eventlet doesn't have into a asynchronous client/server library for python 2.6

 

[Rexibit]

Found this interesting article while researching Syncless. pts.blog: Emulating Stackless Python using greenlet

He's done a lot of bug fixes to syncless since that article was written, but I don't see any complete rewrites of the library like one of the commenters suggested to resolve unnecessary communication.

 

[tdfxx]

Eventlet is the shit. You can write a synchronous function for scraping, see if it works, and then add ~5 lines of code from Eventlet to make it asynchronous. No thread locks and syncing, threadsafe queues, or debugging headaches. It's one of those things that really surprises me for not being more popular in the Python community.