# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # Completely block the Exalead Exabot, see: http://www.exalead.com/software/services/knowledgebase/websearchengine/webmasterguide/ User-agent: Exabot Disallow: / User-agent: * Sitemap: http://www.dealbase.com/sitemap_index.xml Disallow: /new$ Disallow: /history/ Disallow: /edit/ Disallow: /revert/ Disallow: /book/ Disallow: /favorites$ Disallow: /hotel_deals/book/ Disallow: /hotels/rate_source/ Disallow: /login$ Disallow: /logout$ Disallow: /user_session/ Disallow: /password_resets Disallow: /alerts/new$ Disallow: /alerts/*/edit$ Disallow: /*.atom$ Disallow: /*/*.atom$ Disallow: /search Disallow: /*discounts-m-* Disallow: /distribution Disallow: /*-hotel-deals-discounts-NA* Disallow: /blog/wp-admin Disallow: /blog/wp-includes Disallow: /blog/wp-content/plugins Disallow: /blog/wp-content/cache Disallow: /blog/wp-content/themes Disallow: /blog/trackback Disallow: /blog/feed Disallow: /blog/rss Disallow: /blog/comments/feed Disallow: /blog/feed/$ Disallow: /blog/*/feed/$ Disallow: /blog/*/feed/rss/$ Disallow: /blog/*/trackback/$