Im a programmer also SEO and my biggest regret is not working on my own sites earlier. SEO for an agency in my experience has been a pain in the ass. A lot of the time if you want to actually get results, the client has to do some work that they rarely ever do. People want results RIGHT FUCKING NOW and even though they know SEO is a marathon, not a sprint, you're going to be doing things, likely, to get results RIGHT FUCKING NOW!$!@$!@.
If I were you, id probably go with software engineer and I've done that job before. I liked it better than SEO too, its alot less stress. But if you do SEO you have a gigantic advantage - knowing how to code. You know what google can and cannot do in a broad sense, which I've found to be a tremendous advantage. But you may encounter bosses that won't let you do what you need to do, or won't let you tell the truth about SEO, which is, SEO is now web presence optimization if you want to do it for the long haul. That means email, social, conversion optimization(this is a big part of SEO nowadays, lots of people just don't know it yet), ppc, offline advertising, everything down to how the client answers and ends phone calls.
One biggie, its easy to get jobs in SEO, and if you can show past performance, you can get a job that pays 100k+, be "the" seo guy at a company and basically be master of something other nerds wish they knew. I think it depends on you as well. Lots of nerds don't have a sense of urgency, they often have no sense of design, the little shit may not matter. If you're good at planning for the future, acting now for gains in 3-6 months, you pay attention to detail, like milliseconds, you enjoy pouring through data and knowing just about all of the ins and outs of web development (required to be a good seo imo), then you may like SEO. SEO's are often tasked with PPC, social media marketing and the other shit that commonly is not considered SEO, and that experience is great, the education is excellent and the client pays for it.
Also, if I had to guess, 90% of SEO companies, if not more, are scam artists, they dont know or give a shit if the client gets any results. Don't work for them, obviously, what you learn will be useless. Ideally you'll work for a place where you handle a client from beginning to end rather than be a widget on in the machine.
Here, have some tits while you make your decision