User-agent: * Disallow: /weblinks/outside/ #don't crawl the broken link url