imacros help

[wickedDUDE]

Out of a long list of urls, I need to know which ones contain a specific string of text on the page. The text is exactly the same on each page.

I've used imacros before to pull other info like meta keywords, but never for text that is mixed in with content.

Thoughts are welcome...

 

[SirKonstantine]

Open scrapebox > import URLS into harvester > open page analyzer add on > add your footprint > press run

 

[wickedDUDE]

Scrapebox is $97, whereas iMacros if free... don't want to spend money on it

 

[wickedDUDE]

Any other thoughts?

 

[cheshire]

grep -r "string to find" /path/to/files

 

[mattseh]

import requests
urls = (l.strip() for l in open('urls.txt'))

for url in urls:
    try:
        r = requests.get(url)
        print url, 'footprint' in r.text
    except:
        pass

 

[ShadowCaster]

import requests
urls = (l.strip() for l in open('urls.txt'))

for url in urls:
    try:
        r = requests.get(url)
        print url, 'footprint' in r.text
    except:
        pass

Dang that's short.

 

[erect]

...
load url
extract html
open tab
submit html to a web form
process said webform for the phrase with php and return what you need
extract html
close tab
....

Same exact principle for firing off a captcha to services really.

 

[microphone head]

Gems like this tiny thread are why I like this place.