Search entire site?

W

wickedDUDE

New member
Jun 25, 2006
1,054
12
0
What is the best way to search an entire site (with 1000s of pages) for a specific keyword?

I am only referring to words that can be found on the page itself. i.e. that the user would see.

i.e. I want a listing of all pages that have the word x on them.
 


google.com
enter
site:yourdomain.com "keyword"

You can use more "keyword2" "keyword3" to narrow down your search.
Thing about google is they can find things in html and javascript code as well, not just the <body> text, but that's not what you're after.

This will rely on google having your all of/most of your thousands of pages in index.

Regarding on-site usability, the site can have a built-in search engine like wordpress does, but some people called it shit, or something like Sphider - a php spider and search engine .
I've never used sphider but someone recommended it a while ago.

There are probably better ways.
 
Thanks - I'm aware of that method already.

It isn't efficient for me because we're dealing with thousands of pages, and Google only shows 10 results per page. I need something that can simply export the list of pages into a list format (preferably excel) showing the number of matches on each.
 
scrapebox v2 custom harvester with the right keywords to use.

site:yourdomain.com "kw1" a
site:yourdomain.com "kw1" b
site:yourdomain.com "kw1" ..
site:yourdomain.com "kw1" z
site:yourdomain.com "kw2" a
site:yourdomain.com "kw2" ..
site:yourdomain.com "kw2" z

There's export function there after the harverster's done to excel,csv,openoffice etc format.
You'll need some time to refine this. Get sb from scrapebox.com/bhw before they hike the price up.
 
wickedDUDE said:
Thanks - I'm aware of that method already.

It isn't efficient for me because we're dealing with thousands of pages, and Google only shows 10 results per page. I need something that can simply export the list of pages into a list format (preferably excel) showing the number of matches on each.
Click to expand...

You could try this:
1. Set the results per page to the maximum of 100.

2. Use gInfinity to load all of the pages into a single page.

3. Use the Simple Google Results bookmarklet to quickly convert the results into a list of URL's.
​
 
You must log in or register to reply here.
Top Bottom