# All robots will spider the domain User-agent: * Disallow: /coupons/moreDeals.html Disallow: /mpofferref Disallow: /mpclick Disallow: /twitterStatus.html