Proxies Needed for the Best Automation Network Ever

cardine

cardine

...
Jan 9, 2008
3,582
77
0
wordai.com
I have an automation network that uses a lot of bandwidth that is currently almost shut down because my IP's are getting banned, so I need a good proxy source. However I can't use a typical proxy provider because my needs are a bit different (see below).

The exact specs I'm looking for are as follows:
- At least 10tb/mo with the ability to scale up to unmetered 100mbps
- ~30 fresh IPs/mo

In terms of prices I was quoted a long time ago at $750/mo for an entire c-class with 100mbps unmetered (unfortunately that guy is MIA). So based on that and the specs I'm looking for I think somewhere around $400-500/mo is reasonable.

If somebody can refer me to a good provider I'll tell them privately one of the sites that I'm automating.

Also it can be a whitehat network or a network that doesn't allow you to do things that will muddy their IP's. The site that I'm automating is a site that no DC would care about getting banned from, and the odds that my IPs would get reported is appx 0%.

I know that proxy threads are created pretty frequently but I've never seen a good thread about high bandwidth proxies.
 


dchuk said:
why not buy a bunch foreign VPS with unmetered bandwidth and run your own proxy software on them? Or do you need American IPs?
Click to expand...

I can run my entire network on 1-2 IP's but my IP's are getting banned almost every day.

So each VPS would only be useful to me for a couple days so I'd be wasting a lot of money.
 
Are you sure you want an entire C-Block? You would probably be much better off with non-sequential IP addresses. You can get those from any old proxy provider (squidproxies.com for example, a bunch of others - just google it), but it is going to be expensive. Honestly, you would probably be much better off buying three VPSes from three different providers, putting 10 IPs on each, and installing Squid (not sure if Squid works on VPSes? Otherwise use a dedicated, it will still be cheaper than $750/mo). When you burn through those IPs, just ask for 10 more at each place. If they don't give them to you, move on.

Hope things are going well at CMU!
 
chatmasta said:
Are you sure you want an entire C-Block? You would probably be much better off with non-sequential IP addresses. You can get those from any old proxy provider (squidproxies.com for example, a bunch of others - just google it), but it is going to be expensive. Honestly, you would probably be much better off buying three VPSes from three different providers, putting 10 IPs on each, and installing Squid (not sure if Squid works on VPSes? Otherwise use a dedicated, it will still be cheaper than $750/mo). When you burn through those IPs, just ask for 10 more at each place. If they don't give them to you, move on.

Hope things are going well at CMU!
Click to expand...

Yeah I don't want an entire c-block. I checked out squidproxies and they look like they are offering exactly what I want (especially since they offer fresh proxies each month). I send them a message about bandwidth and if they respond back saying they can handle 100mbps I think I'll go with them.

I just don't really want to host it myself because I don't want to have to manage buying/setting up new proxies/ips every month, I want that to be somebody else's problem. But if I absolutely have to I will.
 
FYI I plan on coming out with a service like SquidProxies at some point (not in the next few weeks or anything). At the moment I am working on a public proxy finder that scrapes multiple private lists as well as a bunch of public ones. I'm shooting for 200 live and very good at any one time. PM me if anyone will be interested in that. Hoping to get it live within the week.
 
Berto said:
Sadly I'd say go troll around every provider at BHW. Squid seems the best for my batch too, but I'm doing 0.000002% of what you're doing.
Click to expand...
I'm checking them out, just so far I'm having trouble finding one that offers enough bandwidth for me. Most of them say unlimited/unmetered but I've yet to find one that will promise me that they can handle 100mbps and for the stuff I'm automating I cannot afford to be throttled or have any downtime.


chatmasta said:
FYI I plan on coming out with a service like SquidProxies at some point (not in the next few weeks or anything). At the moment I am working on a public proxy finder that scrapes multiple private lists as well as a bunch of public ones. I'm shooting for 200 live and very good at any one time. PM me if anyone will be interested in that. Hoping to get it live within the week.
Click to expand...
PM'd

-joe- said:
What's wrong with renting a linux dedi, installing squid on it, and buying a c-block (or less) from the dedi provider?
Click to expand...
Because the site I'm automating will just ban the entire c-class.

I'm pushing 100mbps... I'm obviously on the radar of the site I'm automating.
 
cardine said:
Because the site I'm automating will just ban the entire c-class.

I'm pushing 100mbps... I'm obviously on the radar of the site I'm automating.
Click to expand...
So is this a smaller site? I use 15 proxies for scrapebox, on this server:

1288848195.png


And haven't got banned from yahoo/google/AOL.
 
-joe- said:
So is this a smaller site? I use 15 proxies for scrapebox, on this server:

1288848195.png


And haven't got banned from yahoo/google/AOL.
Click to expand...

lol.

First thing: webair has dirty ip's. Not that I care, but just letting you know.

Second thing: How the fuck is 1tb/day (which is pretty much the same thing as unmetered 100mbps) a small site? There's a difference between 100mbps speed on a shared line (where you either get throttled or capped) and 100mbps on a dedicated line.
 
cardine said:
lol.

First thing: webair has dirty ip's. Not that I care, but just letting you know.

Second thing: How the fuck is 1tb/day (which is pretty much the same thing as unmetered 100mbps) a small site? There's a difference between 100mbps speed on a shared line (where you either get throttled or capped) and 100mbps on a dedicated line.
Click to expand...
1. kk, thanks for letting me know :), it's not too much of a problem for me though, I use proxies, so not a problem.

2. You said 10tb a month, so 300gigs a day? That's obviously not small when you compare to somewhere like wickedfire for example, but for major major sites (I'm thinking ehow, ezine, hubpages, squidoo, warez-bb), it'd likely just be a blip. And that speedtest was done by myself.
 
cardine said:
I have an automation network that uses a lot of bandwidth that is currently almost shut down because my IP's are getting banned, so I need a good proxy source.
Click to expand...

You're doing it wrong.

Eventually you'll run out of IPs you can use and make this revenue stream useless to you, assuming that source you're using the IPs for just keeps banning and leaving them. You need to be fixing how you're alternating proxies and get better at avoiding detection.

Surely keeping up a pool of 100+ proxies, rotating more frequently, and changing your process to look much more natural is worth the amount of time you're investing in always searching for a new proxy source.
 
Rexibit said:
You're doing it wrong.

Eventually you'll run out of IPs you can use and make this revenue stream useless to you, assuming that source you're using the IPs for just keeps banning and leaving them. You need to be fixing how you're alternating proxies and get better at avoiding detection.

Surely keeping up a pool of 100+ proxies, rotating more frequently, and changing your process to look much more natural is worth the amount of time you're investing in always searching for a new proxy source.
Click to expand...
Unfortunately that won't work either.

I got to see how the site I'm automating bans users (because their owner is an idiot) and it won't matter either way. When you are running a ton of bots per IP they will give you captchas (which are no problem), but I don't think they ban your IP.

They decide how to ban your IP by manually looking at 1 out of every X submissions and marking it either as 'spam' or 'not spam'. So it doesn't matter how many IP's I spread it across, the amount of volume I'm sending will determine how often my IP's will get banned.
 
cardine said:
Unfortunately that won't work either.

I got to see how the site I'm automating bans users (because their owner is an idiot) and it won't matter either way. When you are running a ton of bots per IP they will give you captchas (which are no problem), but I don't think they ban your IP.

They decide how to ban your IP by manually looking at 1 out of every X submissions and marking it either as 'spam' or 'not spam'. So it doesn't matter how many IP's I spread it across, the amount of volume I'm sending will determine how often my IP's will get banned.
Click to expand...

then just automate deployment and tie into amazon/rackspace/linode apis and start raping servers. Once denied, have your system spin up a whole new server. Have more than running at once so you have less downtime when one needs to move.
 
dchuk said:
then just automate deployment and tie into amazon/rackspace/linode apis and start raping servers. Once denied, have your system spin up a whole new server. Have more than running at once so you have less downtime when one needs to move.
Click to expand...

I'm already doing that.

But I'm worried Rackspace will start to get suspicious if everyday I delete 20-30 cloud servers and create 20-30 new ones (which is the scale I'm working at). My pattern of automatically deleting/deploying servers is pretty much telling them "I am systematically dirtying your IPs" (I'm not, but it would definitely look like it from their end).

Right now, I am doing this with 1-2 cloud servers per day, but I'd much rather make $2000/day off this network than $100/day.


If you think that Rackspace won't care that I'm creating/deleting 20 new cloud servers per day that'd be great for me and I'll do exactly what you said and not worry at all about proxies.
 
I don't see why they would be that upset about it, I mean, that's why they have their API. What you should do is do bursts of scaling to simulate a site under heavy load from time to time.

Make your servers look like their getting Dugg all the time.
 
dchuk said:
I don't see why they would be that upset about it, I mean, that's why they have their API. What you should do is do bursts of scaling to simulate a site under heavy load from time to time.

Make your servers look like their getting Dugg all the time.
Click to expand...

What I might do is whenever my IPs get banned, create 20 more, set those up and then wait another day to delete the banned accounts.

That makes it look like I'm scaling up to 40 temporarily then scaling back down to 20.

It'll look a whole lot more natural than me deleting 20 and then immediately creating 20 more (which is what I was doing, and doesn't look natural at all).

All of my cloud servers run at 100% CPU Usage all the time so I won't have any difficulty simulating heavy load :)


I'll just have to hope they don't do a manual review of my account, because it'll be obvious to them what I'm doing.
 
cardine said:
I'm already doing that.

But I'm worried Rackspace will start to get suspicious if everyday I delete 20-30 cloud servers and create 20-30 new ones (which is the scale I'm working at). . .
Click to expand...
Sounds like this is using up significant amount of your time with a lot of stress and energy thrown in. What if you approached "amazon/rackspace/linode" or other and said this is what I require . . . What can you offer me for my business with the stipulation that I never have to worry about being shut down, throttled, or any issue within this scenario? (define scenario)

cardine your description of volume truly sounds god like. Make the vendors subjugate to your whims as the minions must in the presence of the lord.
 
You must log in or register to reply here.
Top Bottom