# Robots.txt file from http://www.netlandcn.com # All robots will spider the domain User-agent: * Disallow: /oldspider/ Disallow: /inc/ Disallow: /asp/ Disallow: /links/ Disallow: /cert/ Disallow: /database/