# Block certain folders from all crawlers User-agent: * Disallow: Disallow: /Hyperlink pages/ Disallow: /image-htm-files/ Disallow: /images/ Disallow: /page-element-htm/ Disallow: /photogallery/ Disallow: /cgi-bin/ # Block Giraf.com from downloading User-agent: User-agent: girafabot Disallow: / # Block Picsearch.com from downloading User-agent: psbot Disallow: / # Block Picsearch.com from downloading User-agent: psbot Disallow: / #Block SiteSell robot User-agent: SBIder Disallow: / #Block SiteSell/Nutch robot User-agent: Nutch Disallow: /