Proxy Goblin - Proxy Scraper with a Sick Automation mode ;)

Status
Not open for further replies.
A

ashbeats

Member
May 18, 2007
150
2
18
linkfuu.com
Rating - 100%
12   0   0
Hey guys,

It's been some time since I've released anything new here since YellowBot last year so I hope you enjoy this. :)



Proxy Goblin
A Proxy Scraper with a Sick Automation mode.

Automated%20Mode.png


Mix & Match tasks like Sending Email with Proxies, Executing Python Commands, Executing MS-DOS Batch Commands, Uploading to Remote FTP Location, Saving proxies to disk.

Just set the minimum proxy level requirements, create a few tasks, set the interval period and let the Goblin run continuously.
:food-smiley-002:

 I’ve also made the software completely customizable.
So you can tweak almost every setting to your liking.

Modify everything from timeouts, judge urls, max connections, etc.

Easily add additional url sources to scrape proxies from, and if you don’t like the inbuilt sources, you can also chose to only scrape your urls.

You can also easily blacklist ip addresses and hostnames using wildcards. Advanced users can use PCRE flavoured regex for more control.


And these are just some of the features that I’ve mentioned.


​
How does it Work?
Schedule an unlimited number of tasks, and let the scraper run continuously. Sit back and let your proxy list grow.

​
How Can I Use it?
An example campaign could go like this but it’s only limited by your creativity. ;)
​

  1. Gather proxies
  2. Scrub & filter
  3. Send email with proxy list
  4. Upload list to FTP Server
  5. Pause for 10 minutes and repeat

 Mix & Match Tasks In Automation Mode

 Email
Send an email to yourself or multiple recipients with the filtered proxies.
​

 Save to Disk
Save filtered proxies to a CSV or simple text file. Placeholders, like %date% can be used in the filenames. If the file exists, you can even choose to append or overwrite.
​

 Save to Remote FTP Location
Same as save to disk but to a remote FTP location. Great when you have a cron on your server that reads the text file. Constantly have fresh proxies.
​

 Execute Python / MS-DOS Custom Commands
There is almost nothing you can’t do with this feature. You can write custom scripts in either ms-dos’s batch flavour or python. Something sweet for advanced users. :jester:
​


How Much? :music06:


 ​
I’m planning to release Proxy Goblin to the open market at $67. But that's going to take some time, so until I get the site up, you can purchase it here for a special discounted rate. There is also no manual yet, but that will be up in a few days time. Till then if you have any problems just give me a buzz.


Use the following discount code to get over 60% off the retail price of $67.

Coupon Code: WICKEDFIRE

Price After Discount:
Only $27.00

[Click Here To Download The Goblin]
* Don't forget to enter the coupon code & hit the "Checkout" button in bottom right corner of the page


All the best and I hope you enjoy using the Goblin!

 Cheers,
 Ash


Additional Screen-shots:


​
Automated Mode
Automated%20Mode.png


Manual Mode
Manual%20Mode.png


Settings - Additional Sources
Settings%20-%20Additional%20Sources.png


Settings - Blacklists
Settings%20-%20Blacklists.png


Taks - Python Commands
Taks%20-%20Python%20Commands.png


Task - Dos Commands
Task%20-%20Dos%20Commands.png


Task - Email
Task%20-%20Email.png


Task - Ftp Proxies
Task%20-%20Ftp%20Proxies.png


Task - Save Proxies To Disk - Filename Placeholders
Task%20-%20Save%20Proxies%20To%20Disk%20-%20Filename%20Placeholders.png


Task - Save Proxies To Disk
Task%20-%20Save%20Proxies%20To%20Disk.png


Tasks Overview - After adding 4 tasks
Tasks%20Overview%20-%20After%20adding%204%20tasks.png


Tasks Overview
Tasks%20Overview.png
 


Looks good, is there anyway I can check if a proxy is banend from twitter?
i.e. try and access the twitter website through a proxy and see what the response from the server is?
thanks
 
deanimus said:
Looks good, is there anyway I can check if a proxy is banend from twitter?
i.e. try and access the twitter website through a proxy and see what the response from the server is?
thanks
Click to expand...

Sure man. That's the power of the custom commands area. Create a save to disk task and a custom python script to manipulate the data.

So your python script will be the one checking it against twitter.

The Goblin Scrapes & Filters Proxies -> Saves it to a file -> Activates Your custom python script.

Cheers,
Ash
 
deanimus said:
Looks good, is there anyway I can check if a proxy is banend from twitter?
i.e. try and access the twitter website through a proxy and see what the response from the server is?
thanks
Click to expand...

garymoveout said:
How to check if proxy is banned in google? For dumb not for programmer of course :)
Click to expand...



If you've already bought the Goblin, do download it again because I've added two extra judges for you.

There are two ways you can verify if a proxy works with twitter/google.

#1 - Using a custom python script in the custom commands task area.

or

#2 - Use one the new php judges I've included. There should be 3 judges included in the download.

(proxy-judge-google.php, proxy-judge-twitter.php and proxy-judge.php. )


I hope this helps. :)

Cheers,
Ash
 

Attachments

  • judge_url.png
    judge_url.png
    3.5 KB · Views: 48
Review:

I was offered a review of this application by ashbeats and have been using it for the last 4 days. First off a note on the author, I contacted him a few times on chat and he was very helpful right from the start. I mentioned one feature request and he said he would get on it right away, he released yellabot here last year that people loved, overall he seems like a nice guy to work with and I don't think support will be a problem with this app.

Installation: The installation on Win7 was quick and easy and worked out of the box. Proxy Goblin even checks the TCPIP connection limit on your machine and lets you know if it needs to be patched (a very nice addition). You install the app, enter your license key and thats it.

Obviously the first job I done with the app was set it to scrape elite proxies using the default settings. I let it run for a while and came back to it later, it had found > 200 working elite proxies, very nice!. I loaded these into scrapebox and found almost none of them were blacklisted (immediately ;) ) by Google. These proxies are perfect for scraping serps, a second run with the filter level just set to 'Anonymous' returned 17676 raw proxies and after checking I had over 1000 working an0n proxies (this is all with the default settings so hosting your own judge with your own sources = pure win).

Although scraping and testing proxies is the main use of this tool it also has some other very neat features. My favorite is the task scheduler that allows you to schedule and run your own custom scripts. For example lets say you have a script written in Python that registers accounts somewhere/views youtube videos etc. You can set this app up to scrape proxies, when its finished it will save them to a file and run your script (which would read the proxies from the file). It can then either scrape again or execute more scripts etc. I didn't use the mail or ftp features of the app but I am sure they work as described. The FTP feature is a nice touch if you have other scripts running on a server that require proxies.

You can specify timeouts and number of threads for both the scraper and proxies individually and there is a nice regex matching feature that allows you to block proxies from specified ip ranges from being returned.

One feature I would love to see implemented is GEOIP of the proxies it finds, I don't know if the author has any plans to include this but it would be a very nice addition.

Overall I can't really find fault with this application, it's easy to use for novices and has some nice advanced features but moreover it does exactly what is says on the tin, recommended.
 
@gutterseo
Thanks Justin. I'm really glad you like the software :)

@ eria86
The app automatically harvests the proxies from various sites and sources. You can also add your own sources to scrape proxies ( ip:port ) from.

Check out this video for a brief overview of Proxy Goblin:
YouTube - Goblin - Overview

I hope this helps man. Do contact me if you need more help.

Cheers,
Ash
 
Excellent software, very happy with my purchase. My suggestion would be to add a scheduler where I can tell it to only do it's thing between certain times of the day, eg. between midnight and 8am, otherwise sit idle. Cheers.
 
Bofu2U said:
Looks like a good package. I like the FTP upload ability - would make it great for having my scripts pull from a file every X minutes and snag working elite's
Click to expand...

brentdev said:
damn... just tried it.. now u can use imacros as well as plugins to auto-import them into scrapebox and senuke x...... smart..now if only there was a mac version
Click to expand...

Thanks! :rainfro: I won't be porting it to mac anytime soon as I'm using plenty of windows api calls. But the Goblin has been tested on iMac with vmWare and it runs beautifully.

Sorry the promotion ended here. I've added a whole lota features and can't update the first post to reflect the changes.

Some of the new features/updates: v1.3.2


  • SeNukeX Plugin
  • ScrapeBox Plugin
  • SickSubmitter Plugin
  • Built in Google Checking
  • Built in Codeen filters ( Separate from blacklists)
  • Email proxies as Zip
Tech Updates

  • Memory fixes in automation mode. The Goblin will use less than 1% cpu when minimized to tray.
  • Judge's Elite & Anon levels has been updated to mimic EPS's level 1 & 2
  • Automatic updates & patching

Still in the works

  • ScrapeBoard Plugin
  • Geo IP Filters

The site will be launched soon, so if you'd like to come on board as an affiliate send me a PM and I'll contact you once it's ready. I'm also game for discussions on closed promotions so give me a buzz at admin|@|molura|.|com

Cheers,
Ash
 
So I paid full price for this and have to say I am beyond disappointed. I purchased it for the sole purpose of running with the box. Every time I harvest fresh proxies in manual mode then set minimum requirement to anon, export to text file, immediately load into the box, and bam nothing but dead/blocked proxies. Unless Ash comes up with some fix I would stay away from this.
 
Well, I've addressed this in the support forum. But I'll attach the thread here as well. :)

Support Group: Proxy Goblin Support | Google Groups

With a bit of settings tweaking, you can get a whole lot more out of the Goblin. Give me a buzz as admin{@}molura{dot}com and I can help you squeeze more juice out of the Goblin. :)


--------------------------------------------------------------------
Brief overview of How ScrapeBox validates proxies.

ScrapeBox has one of the strictest proxy validation requirements. It does not allow you to use your own proxy judges and if you need proxies for scraping the Google,
it only works if the page google.com does not not give a 302 message, or if it
redirects to a non-us google site.

The following tips will help you to increase the number of working proxies in ScrapeBox

 In the Goblin
  1. Use the Goblin's strict judge => http://molura.com/proxy-judge-strict.php
  2. If you need proxies for scraping Serps, check the "Perform Google Verification" checkbox.
  3. Choose Only Elite proxies
In ScrapeBox
  1. Use a low value for the Max Connections for the Proxy Harvester

    Menu >> Settings >> Adjust Maximum Connections >> Proxy Harvester : Set to max 5

  2. Change the proxy harvester's timeout settings to a high value. This might make SB run slightly slower, but it will make sure all working proxies are used.

    Menu >> Settings >> Adjust Timeout Settings >> Proxy Harvester Timeout : Set to above 50 seconds

  3. And finally when testing the proxies, skip the google test, if you're only going to use the proxies for posting and not scraping.
Using the above setting, you should see a marked improvement in the number of valid proxies.

I also ran this by ScrapeBox's developer and this is the response I got:


ScrapeBoxResponse.png


This is the email I sent them:

... many users have been having problems with ScrapeBox flagging multiple Goblin proxies as invalid. For some reason my proxies pass with other testers like EPS & Charon but almost always fail on ScrapeBox.

I did some testing and came up with this response.
https://groups.google.com/d/topic/proxy-goblin-support/tX7mGxyj0_E/discussion

Do let me know if I was completely off and if there are other variables to consider when preparing proxies for ScrapeBox.

Thanks!

Cheers,
Ash
Click to expand...





 
Unfortunately what I plan on using the proxies for is scraping G, not so much the posting so skipping that step isn't really an option. I know they are a pain in the ass, but for me getting at least 10-15 working proxies doesn't seem like too much to ask for.

I tried running on both elite and anon and ended up with the same results unfortunately. The settings were set to the recommended settings, so not much to tweak there, though I will try to play with it a bit.

Sorry for coming across a bit abrasive, just a touch disappointed after reading all the glowing reviews and having the goblin work great the first time I ran it. The past few days tough seem to be a bit rough.
 
Status
Not open for further replies.
Top Bottom