Google Indexing Pages Disallowed in Robots.txt -- Help?

[Enigmabomb]

Ok, I realize I'm outing the fuck out of myself here, but the problem is happening on multiple sites, and without an example you guys may not be able to help me fix it, so the evil you know... blah blah blah.

If I do a query of "site:BestDryShampoo.net" it shows the links that I am using to forward to wherever it is they go. However, in my robots.txt I have disallowed the outbound directory. WMT Confirms this. SO WHY THE FUCK ARE THEY GETTING INDEXED AND MAKING ME LOOK LIKE A CLOAKING SHITFUCK?

Any light here would be appreciated.

Thanks.

Josh

 

[coffeebean]

Supposedly they'll remove those urls completely if you let Google crawl them (unblock with robots.txt) but then have the page return with a meta noindex tag.

Learn more about robots.txt

 

[Rexibit]

SO WHY THE FUCK ARE THEY GETTING INDEXED AND MAKING ME LOOK LIKE A CLOAKING SHITFUCK?

I stopped using Robots.txt for this same reason. Only a couple Search Engines make an attempt at honoring what's listed there.

If you are serious about blocking a directory, then you turn off indexes using .htaccess. If you are serious about stopping a web page from being indexed, then you turn it off using the meta tag to say "NoIndex, NoFollow".

So, add this to those pages on your site:

<meta name="Robot" content="NoIndex,NoFollow" />

 

[Enigmabomb]

The pages in the directory don't go to my site.

NawIMSayeeng?