Note that there are some explanatory texts on larger screens.

plurals
  1. PODuplicated content in Google. SEO for Drupal
    primarykey
    data
    text
    <p>I have a Drupal site that is up and running. The site is not properly optimized for SEO and there is lot of duplicate content that gets generated in google because of the /category, /taxonomy etc</p> <p>The structure is:</p> <p>/var/www/appname/ This contains a custom built application /var/www/appname/drup This contains my drupal installation</p> <p>I went through the site results in a google search site:appname.com and was that there is lot of duplicated content because of /content, /taxonomy, /node etc.</p> <p>My ROBOTS.txt .. in /var/www/appname has the following already in, but I am surprised that the pages are still getting indexed. Please advise.</p> <pre><code>User-agent: * Crawl-delay: 10 Allow: / Allow: /drup/ # Directories Disallow: /drup/includes/ Disallow: /drup/misc/ Disallow: /drup/modules/ Disallow: /drup/profiles/ Disallow: /drup/scripts/ Disallow: /drup/themes/ # Files Disallow: /drup/CHANGELOG.txt Disallow: /drup/cron.php Disallow: /drup/INSTALL.mysql.txt Disallow: /drup/INSTALL.pgsql.txt Disallow: /drup/install.php Disallow: /drup/INSTALL.txt Disallow: /drup/LICENSE.txt Disallow: /drup/MAINTAINERS.txt Disallow: /drup/update.php Disallow: /drup/UPGRADE.txt Disallow: /drup/xmlrpc.php # Paths (clean URLs) Disallow: /drup/admin/ Disallow: /drup/comment/reply/ Disallow: /drup/contact/ Disallow: /drup/logout/ Disallow: /drup/node/add/ Disallow: /drup/search/ Disallow: /drup/user/register/ Disallow: /drup/user/password/ Disallow: /drup/user/login/ # Paths (no clean URLs) Disallow: /drup/?q=admin/ Disallow: /drup/?q=comment/reply/ Disallow: /drup/?q=contact/ Disallow: /drup/?q=logout/ Disallow: /drup/?q=node/add/ Disallow: /drup/?q=search/ Disallow: /drup/?q=user/password/ Disallow: /drup/?q=user/register/ Disallow: /drup/?q=user/log </code></pre>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload