StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<blockquote> <p>Or would there be some way for people to make $_SERVER['HTTP_USER_AGENT'] appear as Google to bypass the security measures?</p> </blockquote> <p>Definitely. The user agent is laughably easy to forge. See e.g. <a href="https://addons.mozilla.org/de/firefox/addon/59" rel="nofollow noreferrer">User Agent Switcher for Firefox</a>. It's also easy for a spam bot to set its user agent header to the Google bot.</p> <p>It might still be worth a shot, though. I'd say just try it out and see what the results are. If you get problems, you may have to think about another way.</p> <p>An additional way to recognize the Google bot <em>could</em> be the IP range(s) it uses. I don't know whether the bot uses defined IP ranges - it could be that that's not the case, you'd have to find out.</p> <p><strong>Update:</strong> it seems to be possible to verify the Google Bot by analyzing its IP. From <a href="http://googlewebmastercentral.blogspot.com/2006/09/how-to-verify-googlebot.html" rel="nofollow noreferrer">Google Webmaster Central: How to verify Googlebot</a></p> <blockquote> <p>Telling webmasters to use DNS to verify on a case-by-case basis seems like the best way to go. I think the recommended technique would be to do a reverse DNS lookup, verify that the name is in the googlebot.com domain, and then do a corresponding forward DNS->IP lookup using that googlebot.com name; eg:</p> <p>host 66.249.66.1 1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.</p> <p>host crawl-66-249-66-1.googlebot.com crawl-66-249-66-1.googlebot.com has address 66.249.66.1</p> <p>I don't think just doing a reverse DNS lookup is sufficient, because a spoofer could set up reverse DNS to point to crawl-a-b-c-d.googlebot.com.</p> </blockquote>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload