User-agent: * Disallow: /cgi-bin/ Disallow: /inc/ Disallow: /electronic-system-for-travel-authorization-files/ User-agent: psbot Disallow: / User-agent: Microsoft URL Control Disallow: all User-agent: Offline Explorer Disallow: all User-agent: linkwalker Disallow: / User-agent: spidersoft Disallow: / User-agent: linksmanager Disallow: / User-agent: pavuk Disallow: / User-Agent: sitecheck.internetseer.com Disallow: /