StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POExcluding bad bots via htaccess: why I get http 500 errors?
primarykey
Id
18949368
data
AcceptedAnswerId
0
AnswerCount
1
ClosedDate
CommentCount
14
CommunityOwnedDate
CreationDate
2013-09-22T22:23:36.333
FavoriteCount
0
LastActivityDate
2014-06-05T03:19:20.900
LastEditDate
2013-09-23T11:43:30.017
LastEditorUserId
1430117
OwnerUserId
1430117
ParentId
0
PostTypeId
1
Score
0
ViewCount
472
LastEditorDisplayName
text
Body
I'm blocking bad and useless bot using: <pre><code>RewriteCond %{HTTP_USER_AGENT} ^$ [OR] RewriteCond %{HTTP_USER_AGENT} 360Spider [OR] RewriteCond %{HTTP_USER_AGENT} A(?:ccess|ppid) [NC,OR] RewriteCond %{HTTP_USER_AGENT} C(?:apture|lient|opy|rawl|url) [NC,OR] RewriteCond %{HTTP_USER_AGENT} D(?:ata|evSoft|o(?:main|wnload)) [NC,OR] RewriteCond %{HTTP_USER_AGENT} E(?:ngine|zooms) [NC,OR] RewriteCond %{HTTP_USER_AGENT} f(?:etch|ilter) [NC,OR] RewriteCond %{HTTP_USER_AGENT} genieo [NC,OR] RewriteCond %{HTTP_USER_AGENT} Ja(?:karta|va) [NC,OR] RewriteCond %{HTTP_USER_AGENT} Li(?:brary|nk|bww) [NC,OR] RewriteCond %{HTTP_USER_AGENT} MJ12bot [NC,OR] RewriteCond %{HTTP_USER_AGENT} nutch [NC,OR] RewriteCond %{HTTP_USER_AGENT} Pr(?:oxy|ublish) [NC,OR] RewriteCond %{HTTP_USER_AGENT} robot [NC,OR] RewriteCond %{HTTP_USER_AGENT} s(?:craper|istrix|pider) [NC,OR] RewriteCond %{HTTP_USER_AGENT} W(?:get|(?:in(32|Http))) [NC] RewriteRule .? - [F] </code></pre> Complete htaccess file: <pre><code>AddDefaultCharset UTF-8 RewriteEngine on #inherit from root htaccess and append at last, necessary in root too RewriteOptions inherit #block bad bots RewriteCond %{HTTP_USER_AGENT} ^$ [OR] RewriteCond %{HTTP_USER_AGENT} 360Spider [OR] RewriteCond %{HTTP_USER_AGENT} A(?:ccess|ppid) [NC,OR] RewriteCond %{HTTP_USER_AGENT} C(?:apture|lient|opy|rawl|url) [NC,OR] RewriteCond %{HTTP_USER_AGENT} D(?:ata|evSoft|o(?:main|wnload)) [NC,OR] RewriteCond %{HTTP_USER_AGENT} E(?:ngine|zooms) [NC,OR] RewriteCond %{HTTP_USER_AGENT} f(?:etch|ilter) [NC,OR] RewriteCond %{HTTP_USER_AGENT} genieo [NC,OR] RewriteCond %{HTTP_USER_AGENT} Ja(?:karta|va) [NC,OR] RewriteCond %{HTTP_USER_AGENT} Li(?:brary|nk|bww) [NC,OR] RewriteCond %{HTTP_USER_AGENT} MJ12bot [NC,OR] RewriteCond %{HTTP_USER_AGENT} nutch [NC,OR] RewriteCond %{HTTP_USER_AGENT} Pr(?:oxy|ublish) [NC,OR] RewriteCond %{HTTP_USER_AGENT} robot [NC,OR] RewriteCond %{HTTP_USER_AGENT} s(?:craper|istrix|pider) [NC,OR] RewriteCond %{HTTP_USER_AGENT} W(?:get|(?:in(32|Http))) [NC] RewriteRule .? - [F] #include caching for images <IfModule mod_expires.c> ExpiresActive On ExpiresByType image/gif "access plus 1 month" ExpiresByType image/jpeg "access plus 1 month" ExpiresByType image/png "access plus 1 month" ExpiresByType image/x-icon "access plus 360 days" ExpiresByType text/css "access plus 1 day" ExpiresByType text/html "access plus 1 week" ExpiresByType text/javascript "access plus 1 week" ExpiresByType text/x-javascript "access plus 1 week" ExpiresByType application/javascript "access plus 1 week" ExpiresByType application/x-javascript "access plus 1 week" ExpiresByType application/x-shockwave-flash "access plus 1 week" ExpiresByType font/truetype "access plus 1 month" ExpiresByType font/opentype "access plus 1 month" ExpiresByType application/x-font-otf "access plus 1 month" </IfModule> RewriteCond %{HTTP_HOST} ^nix.foo.com$ [OR] RewriteCond %{HTTP_HOST} ^www.nix.foo.com$ RewriteRule ^(.*)$ "http\:\/\/www\.foo\.com\/nix\.php" [R=301,L] RewriteCond %{HTTP_HOST} ^gallery.foo.com$ [OR] RewriteCond %{HTTP_HOST} ^www.gallery.foo.com$ RewriteRule ^(.*)$ "http\:\/\/www\.foo\.com\/gallery\.php" [R=301,L] RewriteCond %{HTTP_HOST} ^blog.foo.com$ [OR] RewriteCond %{HTTP_HOST} ^www.blog.foo.com$ RewriteRule ^(.*)$ "http\:\/\/www\.foo\.com\/blog" [R=301,L] RewriteCond %{HTTP_HOST} ^id.foo.com$ [OR] RewriteCond %{HTTP_HOST} ^www.id.foo.com$ RewriteRule ^/?$ "http\:\/\/foo\.myopenid\.com\/" [R=301,L] redirect 301 /map.php http://www.foo.com/maps/map.php RedirectMatch 301 ^/(map(?!pa_area51\.)[^/.]+\.php)$ http://www.foo.com/maps/$1 Options +FollowSymLinks RewriteCond %{HTTP_HOST} !^www\. RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L] </code></pre> It worked good (http 403) until I switched from a Litespeed webserver hosting to an Apache's one. They're both shared hosting services. Now I get: <pre><code>Forbidden You don't have permission to access /robots.txt on this server. Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request. </code></pre> Here's a sample from access log: <pre><code>208.115.111.68 - - [22/Sep/2013:17:56:48 +0200] "GET /robots.txt HTTP/1.1" 500 576 "-" "Mozilla/5.0 (compatible; Ezooms/1.0; ezooms.bot@gmail.com)" </code></pre> Any hints on that http 500 error? Thanks in advance
Tags
<apache><.htaccess>
Title
Excluding bad bots via htaccess: why I get http 500 errors?
singulars
PostAcceptedAnswerId
1. This table or related slice is empty.
PostParentId
1. This table or related slice is empty.
PostTypePostTypeId
1. PTQuestion
UserLastEditorUserId
1. USflapane
UserOwnerUserId
1. USflapane
plurals
PostLinksPostIdRelatedPostId
1. This table or related slice is empty.
PostLinksRelatedPostIdPostId
1. This table or related slice is empty.
PostsAcceptedAnswerId
1. This table or related slice is empty.
PostsParentIdCreationDate
1. This table or related slice is empty.
VotesPostIdCreationDate
1. This table or related slice is empty.
CommentsPostId

Querying!

Guidance

A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.