Sitemap (robots.txt) problems

Best WordPress theme for game news websites, game portals or whatever your preferred subject.
GK User
Wed Jan 02, 2013 9:39 am
Hi,

I have come accross with problems regarding my virtual robots.txt file with this template. My virtual robots txt looks:

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

Sitemap: http://mydomain/sitemapindex.xml

I swtiched off the SEO options of the theme, and Google Webmaster says upon uploading the sitemap that robots.txt disabled some URLs. :( It is just a warning, not an error.

I use a third party plugin for generating the sitemap. I tried it with two different plugins, in each cases I receive the same warning.

On the same server I have another domain, with the same plugin installed, it works perfectly. This is why I think it can be related to this theme.

Any idea what to do? Thanks
User avatar
Senior Boarder

GK User
Wed Jan 02, 2013 9:50 am
Hi,

That's very strange, because our SEO options are connected only with the keywords/description metatags and with the page title.

Are you sure that the problem isn't connected with the "Disallow" rules in your robots.txt file?
User avatar
Administrator

GK User
Wed Jan 02, 2013 10:02 am
dziudek wrote:Hi,

That's very strange, because our SEO options are connected only with the keywords/description metatags and with the page title.

Are you sure that the problem isn't connected with the "Disallow" rules in your robots.txt file?


I have got only two disallow functions in the virtual file (it is not a real robots.txt, the sitemap plugin generates it).

Disallow: /wp-admin/
Disallow: /wp-includes/

Nothing more. Do you think I should create a manual robots txt file?
User avatar
Senior Boarder

GK User
Wed Jan 02, 2013 10:08 am
Do you know which URLs has been disabled?
User avatar
Administrator

GK User
Wed Jan 02, 2013 10:14 am
dziudek wrote:Do you know which URLs has been disabled?


Here some examples:

http://www.ouyahungary.hu/
http://www.ouyahungary.hu/a-hawken-is-m ... -konzolra/

In fact, all URLs on the site.
User avatar
Senior Boarder

GK User
Wed Jan 02, 2013 10:20 am
BTW did you disabled this option: "Discourage search engines from indexing this site" in the Reading Settings?
User avatar
Administrator

GK User
Wed Jan 02, 2013 10:22 am
Of course, this option is disabled.
User avatar
Senior Boarder

GK User
Wed Jan 02, 2013 10:25 am
Very strange. In this case I think that you should create manually the robots.txt file.
User avatar
Administrator

GK User
Wed Jan 02, 2013 10:26 am
dziudek wrote:Very strange. In this case I think that you should create manually the robots.txt file.


I did. Same result. Even if I create manually, the same warning is displayed in Webmaster. :/ I'm totally lost.
User avatar
Senior Boarder

GK User
Wed Jan 02, 2013 10:30 am
Sorry but probably we won't be able to help, because as I wrote earlier - our themes don't operate on the more complex SEO options especially this connected with the indexing, so the problem can be connected with the used SEO plugins or the WordPress settings.
User avatar
Administrator

GK User
Wed Jan 02, 2013 10:32 am
dziudek wrote:Sorry but probably we won't be able to help, because as I wrote earlier - our themes don't operate on the more complex SEO options especially this connected with the indexing, so the problem can be connected with the used SEO plugins or the WordPress settings.


Thanks, anyway.
User avatar
Senior Boarder


cron