User-agent: grub-client Disallow: / User-agent: * Disallow: /500.shtml/ Disallow: /400.shtml/ Disallow: /401.shtml/ Disallow: /403.shtml/ Disallow: /404.shtml/ Disallow: /c/ Disallow: /cgi-bin/ Disallow: /i/ Disallow: /x/ Disallow: /pdo/ Disallow: /way/ Disallow: /comments-giver.php/ Disallow: /pdo.html/ User-agent: googlebot Disallow: /500.shtml/ Disallow: /400.shtml/ Disallow: /401.shtml/ Disallow: /403.shtml/ Disallow: /404.shtml/ Disallow: /c/ Disallow: /cgi-bin/ Disallow: /i/ Disallow: /x/ Disallow: /pdo/ Disallow: /comments-giver.php/ Disallow: /pdo.html/