Setting up AWS S3 (static hosting only) was simple, and I figured it out quickly since I already knew the interface. Their DNS system was a bit trickier, unintuitive process. This blog post covers it all pretty much, though:
Static Web Hosting With Amazon S3 | Chad Thompson
I used a tutorial to set up EC2 for my web apps though, but I picked it up quick since I know my way around SSH. EC2 is basically like having your own remote machine, given to you raw - you install your web server (nginx, lighttpd, or apache), then mysql, php, ruby and rails, etc. I can't find the link I used, but here's another good one that goes through the steps setting up a fresh instance and installing WP.
Setting Up WordPress on Amazon EC2 in 5 minutes | Christophe Coenraets
As for converting my WP sites to Jekyll, that was a longer process that involved alot of Google/Stack Overflow searches. Something I didn't do the first time around, that I recommend doing, is installing the
Disqus Wordpress Plugin to quickly export your comments from WP into Disqus before making the switch (assuming you plan to use Disqus after going static of course.)