Quickie Tidbit: Disallow Search Indexing

Posted byDavid WeeklyPosted inGeneral

Some people who run wikis that are publicly accessible have asked us for the ability to set a robots.txt that tells polite search engines like Google, MSN, Alexa, and Yahoo to please not index the content of the public PBwiki.  We listened and added this functionality. If you’d like to enable this on your PBwiki, just go to the “Public / Private” settings tab and click Disable indexing. This sets your robots.txt, updates your page headers, and even puts a setting in your RSS & Atom feeds to discourage feed readers & scrapers from republishing your public feed.

Published by David Weekly

I like starting things. I founded PBworks (originally PBwiki) and am the cofounder of SuperHappyDevHouse and Hacker Dojo. I advise a dozen startups and am a mentor for Founders Institute, i/o Ventures, and Women 2.0.

4 thoughts on “Quickie Tidbit: Disallow Search Indexing”

  2. That’s correct, private wikis are inaccessible to search engines AND are marked to disallow indexing. (But without the password, the search engines could not even access the page, much less index it!)

    Reply

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.