Yandex does not index my site...
This phrase can now be found on every forum. Both me and thousands of other owners of new sites are very interested in the question: “Why has Yandex been indexing new sites very slowly lately or not indexing them at all?”
Most novice webmasters describe approximately the same problem. Very often, a site is successfully indexed by other search engines, such as Google, Rambler, Yahoo, MSN, Aport, etc., but Yandex flatly refuses to index the site even if there are external links already indexed by Yandex itself. This is the paradox...
And the owners of newly created resources begin to scour the forums in the hope of finding an answer to this difficult question.
By the way, I am no exception, which is why I decided to raise this topic. My site was fully indexed by the Google search engine just two weeks after it was published on the Internet. At that time, the site did not have a single external link, but was added to the webmasters panel.
Yandex has not indexed a single page yet, although more than two months have already passed. And this situation is observed among many site owners. Many of them, tired of finding out the reasons for the lack of indexing on the part of Yandex, began to promote their sites under Google. I feel like I will have to go this route too. Moreover, all new articles on my website are indexed very quickly by Google.
I can say the same about the Rambler search engine. Yes, it is somewhat inferior in terms of indexing speed to Google, but nevertheless, almost all pages of my site have been indexed by it today. It’s a pity that in terms of popularity among Internet users, Rambler lags significantly behind Yandex...
For those who have not yet lost hope of getting into the Yandex index, the next part of the article is devoted to where we will try to find out the main reasons for the absence of a young site in the index.
So, first, let's make sure that Yandex really hasn't indexed a single page of the site. To do this, we will use the advanced search of Yandex itself.
Leave the "I'm looking for" field on the advanced search page empty. In the "On site" field, enter the address of your site and click "Find" at the very bottom of the page. If you receive in response “The combination of words you are looking for does not appear anywhere,” then indexing has indeed passed you by.
Or maybe Yandex doesn’t even know about the existence of your site. Have you added it to the webmasters panel, is it displayed there?
If data about it appears in the webmaster panel, then Yandex is aware and your site is probably stuck in the indexing queue. If the site is not in the webmasters panel, you may not have been able to confirm the rights to it. Try confirming your rights again.
Many webmasters have concerns that their site has been banned or is subject to penalties. It’s easy to check your site for a ban; to do this, add the address of any internal page of your site to the form for adding new URLs (popularly called addurilka). If the result is: “Your address has been successfully added and will be indexed...”, then there is no question of any ban.
If the answer is: "URL is unavailable", then there are technical problems server. To verify this, look at the server logs. Typically, hosting clients have free access to server logs in their admin panel. If there is no access, you can contact hosting technical support with a request to provide the latest entries in the log log.
The log entry looks like this:
275.169.105.78 - [ 29.11.2010: 15:35:47 + 0500 ] " GET/HTTP/1.01 200 OK "_" "Yandex/1.01.001 (compatible, win16, I).
Now let's decipher this entry:
275.169.105.78 is the IP address from which visitors or bots come to you;
11/29/2010: 15:35:47 - this is the date and time they visited your site;
HTTP/1.01 200 is the server response, the server response must contain the number 200, this means that the site page is available for reading. If there is no such response, it makes sense to contact the hoster with a request to troubleshoot;
Yandex/1.01.001 (compatible, win16, I ) is the name of the Yandex bot, pay attention to the last character I , it is inherent in the main Yandex bot. If there is such an entry in the log, then there is no need to panic. This means that the main Yandex bot is visiting your site and perhaps soon the site’s pages will appear in the index.
The main thing is that after Yandex/1.01.001 (compatible, win16, I ) there was no error message, for example HTTP/1.1.404 Not Found, meaning that the server bot did not allow the Yandex bot to read the page. If you find such an entry, contact the hoster immediately.
You can also check the server response in the servicebe1.ru/stat/ or its analogueseo-rus.com. At the end page, after analyzing the site there will be a record like this:
Heading
HTTP/1.1 200 OK
Server: nginx/0.7.67
Content-Type: text/html
Connection: close
ETag: "5c8e26-6636-4961a4c921a80"
Accept-Ranges: bytes
Content-Length: 26166
Vary: Accept-Encoding
In the line after 271 you see the required server response. If, as a result of your inspection, no technical reasons have been identified and your site is not banned or visited by bots, you have not violated any rules for the licensed use of the Yandex search engine, your site is an SDL, the content on the site is unique, you do not trade links, and do not create spam, then The only way out of this situation is to contact Yandex technical support.
When promoting a website to search engines, sometimes the following problem arises - some pages of the resource cannot be indexed by search engines. Why is this happening? It is difficult to answer this question unambiguously until it is possible to look at the project itself and try to correct obvious problems, if any. And even if the reason why search engines ignore a page is found, there is always a possibility that it may not be the only one. Naturally, it turns out this way - if some of the pages optimized for certain keywords are not indexed by search engines, then effective website promotion is simply impossible.
If problems arise with indexing, it initially makes sense to check in each search engine separately selected for promotion - it is quite possible that there is a problem with only one search engine. In this case, the reason may even be a simple mistake. Many systems provide the ability to check whether a certain page is indexed using special operators, but there is a simpler and quick way- just enter the page address in the search bar and see the results. If the search engine does not provide a link to this page of your site, it is logical to assume that it is not indexed.
When you have already realized that a certain page is indeed not indexed in the search engine, you need to check all other pages of the site by entering “site:site_name.ru” into the search bar. At the same time, you will understand exactly which pages of your resource have already been indexed and which are not - after all, the problem can be much larger than one page “forgotten” by search engines. The main causes of problems with indexing, Newmann Bauer experts include:
1. The dynamic page address is too long. Search engines sometimes “refuse” to index pages whose address is too long and contains many additional parameters;
2. If the site pages contain texts that constitute ordinary spam. This can also happen due to oversaturated content keywords. The solution is to fill the page with other content and ask for the page to be re-indexed. Otherwise, spam can ruin the rating of the site as a whole;
3. If the robots.txt file containing instructions for search robots is not compiled correctly, some pages may simply be excluded and there is no need for them to be indexed by search robots. This may also be the reason why a particular resource page is not indexed;
4. Your project contains hidden links. If a project is navigated using non-HTML methods, search engines may simply not see the pages that need to be indexed.
It also happens that all pages are indexed correctly, but one single page remains inaccessible to detection through search engines. In this case, it is quite possible that such a page is available only to those users who fill out a certain form or click on a pop-up window - search robots have not yet learned how to do this. The page may also have a JavaScript redirect or a meta update that prevents indexing. Also, if a frame is used on the page, there are serious errors in the code, or the dimensions are too long, the indexing may well be incorrect.
Hello friends! Today I couldn’t get around the topic of why the site is not indexed in Google and Yandex search engines, since many novice webmasters have this error. Many people were interested in creating their own web page on the Internet, some used free servers, and some were already paid, but these were a few out of millions.
For example, your neighbor made a website on a free one, and you on a paid one, but your site is not indexed, and every day he has a new page in the search. So, the basic rules by which the Yandex robot, google, mail, rambler index the site.
Many will say that you will never know the algorithm of search robots and how they add pages to their search after indexing your site, but I will say right away that it is impossible to say for sure, but you can make an assumption based on a number of rules.
So, the search of any search engine is aimed at high-quality search results for information, that is, the robot should strive for the most similar search results to what the client requested.
Let's make the first conclusion, why your neighbor has fast site indexing, but you don’t. The neighbor uses a seemingly standard template, etc., we can safely assume that he writes regularly and his material is attractive to search engines. It is worth noting that machine robots like to read new things, not old ones.
Search companies have recently written more and more often that they prefer sites (SDL) made for people, and they artificially lower satellites in search results.
As can be seen from the above In order for your site to undergo rapid indexing, you need to regularly update and add relevant, in-demand material to your site, then the robot search engines will visit you more often.
Unfortunately, in order for the site to be indexed successfully, the above is not enough; first of all, you must allow the search robot to index your site. Here are the seven most common problems, why the site is not indexed by Google and Yandex search engines.
7 secrets: why the site is not indexed in Yandex
1. The site is closed with tags noindex, noffolow, incorrect robots.txt
For site indexing to be complete and painless, you need to go to the admin panel - settings - read ( uncheck) ask search engines not to index the site. I wrote how to write the correct robots.txt for a WordPress site.
2. Search robots do not know about the site.
3. The site contains 100% copy-paste.
This means you are not a good person." radish“, copying someone else’s material on the Internet is extremely bad and search engines are well aware of this.
Be sure to write your own text; if you have no imagination, then read and write summaries like in school - this is also a kind of unique content.
4. The content of the site is not meaningful (irrelevant).
It very much depends on the content of the post (text), the longer and more interesting it is, the more chances you get to the top of search engine results.
By irrelevant content, I mean outdated information, for example, about the score between two football teams that played each other a year ago, although on the day of the game the information was on the front pages of newspapers, magazines, and search engines, but today everyone has forgotten about it.
5. Website design.
It would seem that what the design of my site has to do with it. Search robots, as a rule, artificially lower your site in search results, since this is again a standard satellite in their eyes, and given the fact that you cannot even design the site, the indexing of the site is correspondingly slow.
6. Website under filter.
In fact, there are two ways out of this situation, the first is to write to the technical support of the search engine, and the second is to create a new domain and forget about the old one forever. I wrote about how to get out from under the filter.
7. Contacting the site’s technical support.
Is it worth contacting technical support if the site is not indexed? Definitely, in order to write to someone, you must wait a few weeks, after which you can write to support, but most likely your problem lies in your site. Be careful before writing to support.
These are just some of the reasons why a site is not indexed in Yandex and Google, apart from the robots.txt file for your site.
Subject to seven basic rules, you can certainly achieve good results.
And I also want to add that for high search rankings, you need to have, domain age, reputation, good statistics when visiting your site. At the stage of completing my next post, as usual, I give the next lesson from my video course, “how can a beginner earn money effectively on a website passively?”
We increase the speed of indexing of our articles
Dear friends, I hope you learned from my lesson today useful information, and you will share it with pleasure with your friends on , all you need to do is click the button of your favorite site located below. I'm looking forward to it in my next lessons. Bye.
Hello everyone friends!
Many SEOs on various forums often ask why Yandex does not index the site. In this article I will try to answer this question in detail and write possible reasons, why did Yandex not like your resource so much that it doesn’t want to index it: smile:.
So let's get straight to the point.
1. The site is very young.
If you have recently started writing articles, then you don’t have to worry, everything is fine with your project. Yandex indexes young resources for a very long time, sometimes you have to wait several months, yes, yes, this happens.
If your site is very young, then it is quite possible that Yandex has not yet found it on the Internet. But you can help him do it. Now there are many ways to speed up blog indexing. You can read about them. If you do everything that is written there, then in a week your site will definitely be indexed in Yandex, and possibly faster.
2. Sanctions from search engines.
If the site is no longer young, and it was in the index, but then suddenly crashed, then, most likely, Yandex did not like something, and it imposed sanctions on it. There are a lot of these sanctions, but the most popular is the AGS filter. Some people ask: " Why Yandex does not index the site except the main page?. If only the main page is in the index, then most likely the site is under the AGS. I wrote about how to check a blog on the AGS and how to get it out of there.
In addition to the AGS, the site could also be banned - this is when all pages are removed from the index.
3. Random indexing ban.
Many beginners may mistakenly prevent a site from being indexed. For example, in the robots.txt file the following line is responsible for prohibiting indexing:
User-agent: *
Disallow: /
If you want your resource to be properly indexed in search engines, then this line should not be in robots.txt.
Besides this, there are other ways in which you can accidentally. If your resource is made on the Wordpress engine, then go to “Control Panel” - “Settings” - “Reading” and check that there is no checkbox next to the inscription “Recommend search engines do not index the site.”
4. Server error.
Sometimes it may happen that a search robot visits a site and is given a 505 error. Of course, he will immediately leave this site, and I think he will not return to it soon: smile:.
If you go to your blog and often see a 505 error, then this is very bad, I advise you to change hosting. There are also many other errors (500, 302), etc. But I think you should not forget about this. Just check the server response, it should be 200 ok. You can check it on this page:
5. Bad domain history.
If you registered a new domain and did not check its history, then this is very bad. It may be that there was a shit site on this domain a long time ago and search engines banned it. Even if you have now created another white and fluffy resource, search engines know the history and therefore may not index it. I recently wrote an article about . You can use this instruction.
6. Non-unique content.
If you start publishing non-unique articles on a young site, then this is very bad; search engines may stop indexing it or index it worse.
Some webmasters believe that nothing bad will happen if the articles are on other sites, the main thing is that visitors like it. But I think that only those people who are too lazy to write think this way. After all, it’s much easier, found it, copied it, pasted it: smile:. You can publish 100 articles in a day, and in a few years there will be a whole Wikipedia.
In general, don't listen to anyone who says you can copy other people's articles. If you want to get large traffic and then earn money on the site, then you need to write articles yourself or order from, but definitely.
From the author: you write useful and interesting content. Or maybe it’s useless and uninteresting - that’s your business. But why is the site not indexed in Yandex for a long time? Let's look at all the main reasons.
Look at this picture. Yes, you've probably seen her more than once. Indeed, everything can be found in Yandex. But what is not indexed will never be found, because the search engine simply does not know about it. And if so, then the issue of indexing is very important for any webmaster, site owner and optimizer. Today I will write about at least a few reasons why you may have problems with indexing.
The first and most likely reason is the age of the site.
The fact is that most webmasters who are concerned about the issue of fast indexing are owners of young sites. I am confident in this because I myself developed two sites from scratch from the very first days of their existence. You need to understand that Yandex treats newly hatched sites with distrust, and one cannot blame it for this - too many GS have appeared.
So, the younger your site, the less chance of quick indexing of site pages. As a rule, you can get it into the index more or less quickly home page, since it has the highest priority, but the rest, in any case, will not get into the index so quickly.
If you want to index at least the main page, add the site to Yandex.Webmaster if you have not already done so. Also add the project url to addurl (Report a new site). After confirming the rights to the site, I also recommend immediately doing the following:
Upload an xml sitemap to the webmaster (this can directly improve indexing, since the search robot will be better aware of all available pages).
Download and check the robots.txt file. It can be checked for errors.
Set up the main mirror of the site so that there are no duplicates with and without www
This is something that will directly or indirectly affect the indexing speed. If you have done all this, all you have to do is wait for indexing, adding all new project urls using the same addurl.
Okay, maybe you've done all this and still can't wait for indexing. Is there really no way to speed up the process? Here you need to understand that much does not depend on you. Yandex indexes much slower than Google by default. Especially young sites. Usually within 1-3 weeks new pages of young sites appear in the index. Consider this time period quite normal.
I just want to convey to you that if the reason for slow indexing is the age of the site, then you shouldn’t worry too much. Over time, your project will grow and turn from a baby into an ordinary Internet resource, well known to search engines. Then the time will come when new materials will fly into the index quickly, within a few days.
I noticed this situation on two young sites that I promoted. When the site's age exceeded 6 months, the rate of entry of a new article into the index increased. According to Yandex itself, the longest indexing period is 2 weeks. If they pass and there is nothing in the index, then there may be other problems or errors. Of course, you shouldn’t wait 2 weeks, but it’s better to check everything as soon as you can. Next I will consider the most common mistakes and factors due to which the site or its individual pages may not be included in the index.
Prohibition of indexing in robots.txt
This is an important and useful file, but there may be errors in it if you wrote the instructions yourself or took them from the first site you came across. The most important thing is that there is no such command: Disallow: /
Also check what specific commands are given for the Yandex bot (user-agent: Yandex). It's unlikely that the problem is in this file, but it would be stupid not to check it.
Using the robots meta tag
By the way, in robots.txt itself you may not find any dangerous commands that would block crawling to the site and its specific page. But in addition to this, there is also a robots meta tag, with which you can block a specific page from indexing, as well as links on it.
< meta name = ”robots ”content = ”noindex , nofollow ”> |
For example, such a piece of code clearly indicates that the page will be closed. Check if you have it somewhere in your header. If you want to close all links, but open the text (then the page will be indexed), use the parameters index, nofollow, and if you want to open everything - index, follow.
Prohibition of indexing in engine settings
As far as I remember, before installing WordPress, you can check the box to not have your site indexed in search engines. This can be useful for the first time on young sites that are under development, so that all sorts of test pages do not end up in the index. You may have checked this box accidentally during installation or later from the settings. In this case, just remove it.
Incorrect use of noindex tag
The noindex tag is a useful tag, but a dangerous one if used incorrectly. It allows you to block text from indexing. That is, the text placed in it will not be perceived by the search engine. Accordingly, if you accidentally wrap the entire article in noindex, it will not be indexed, since there is nothing to index (everything is closed).
You can hardly make such a mistake, but the tag will creep into the template files themselves. Perhaps it opens somewhere in your header.php, and then does not close anywhere. It turns out to be a funny situation that everything is closed. In other words, check the code of your site. In fact, you don’t have to be a programmer or layout designer to do this.
You can use a program like Archivist to quickly search through many files (type noindex), or use a standard file search. The debugger can also help (F12 in Google Chrome), but this is more for developers. Additionally, you can check each specific page. Did it happen that you wanted to close some non-unique piece of text using noindex, but did not close the tag in the right place?
Search engine filters
In this case, we are interested in filters from Yandex. Although you should understand that Google also has them, and they may also be the reason why your site is not included in the index in this search engine.
In particular, all young sites have a sandbox filter. This is not a problem, this is normal. Just as little children play in the sandbox, so young and green Internet projects also end up there. No, you don’t need to go around the whole city looking for your site, just know that the indexing speed of young sites is lower. I have already spoken about this above.
But the sandbox is a temporary measure for the search engine. And in general, this is not a terrible filter, because it applies to everyone. Another thing is sanctions for using unfair methods when promoting a site. Most often this is hidden or obvious spam using keywords in the text, as well as in meta tags. In other words, play by the rules of search engines, and you won’t have such problems.
Bad domain history
This point is relevant for those who bought a domain and started developing their new project on it, or for those who registered a domain abandoned by the previous owner.
Of course, before purchasing any domain, I recommend that you do your research. This can be partially done using services such as WebArchive (site snapshots) and linkpad.ru (link mass analysis). From the pictures you can determine what the topic of the site was and what kind of content was on it. You can also roughly understand the topic from the reference mass. Also check the domain for search engine filters.
In general, this little analysis will save you from buying a domain with a bad history. If you nevertheless purchased one, perhaps with the help of requests in technical support. support will be restored over time. But it’s better to start from scratch than like this.
Site language
You must understand that Yandex is primarily focused on the CIS countries. If you want the pages of English, German and other foreign sites to be indexed, then this is quite possible, but only if quality content. For foreign sites there is Google and many other search engines, so that's your first place.
Percentage of uniqueness of your texts
There's another one good reason, why your site may not be indexed by Yandex. This reason is non-unique content. For example, you copied text from another source and pasted it into your own without changing it in any way. Practice shows that Yandex may not index completely non-unique content at all and may not pay attention to pages whose uniqueness is less than 50% for a long time. It is better to have at least 90% uniqueness.
To better understand why non-unique content has a much lower chance of being indexed successfully, you need to roughly understand the algorithm by which new pages get into the index. In particular, before hitting the robot, some analysis of the content is carried out and if it turns out to be of low quality, the page simply may not be included in the index. In order for it to get through, it is necessary to improve the quality of the text.
Server errors
This is the last reason we will look at. It is purely technical. In particular, one of the indicators of the quality and reliability of your resource is that it is constantly available online 24 hours a day, every day. Of course, there may be minor glitches and that's normal. But when your site goes down for several hours (or, even worse, days), the search robot, having arrived at new pages, simply will not be able to open them and will be forced to leave.
Thus, those pages that were already there may even fall out of the index. How to prevent all this from happening? First of all, choose a good hoster and a more or less normal tariff. There is no point in saving money on your website. It’s better to pay 300-500 rubles a month than 50.
Also, if you buy VPS/VDS plans, then you will most likely have to configure the server yourself. In particular, choose the OS of the virtual machine, install some utilities, etc. If you are completely new to server setup, it is better not to buy VDS yet or choose a tariff where you are guaranteed technical support. support from the provider.
In particular, server errors include all codes 4**, 5** (three digits with a four and a five at the beginning). For example, due to lack of optimization of database operation, an error may occur. Especially if there are dozens or hundreds of people walking around the site, and you have few resources.
Bottom line
Well, we've covered all the main reasons. Now you know what to do when a site is not indexed in Yandex. If you have checked absolutely everything and found no errors or problems, then you just need to wait. As I already said, Yandex does not index as quickly as Google. The speed of entering the index ranges from 1 day to 3 weeks.
Well, you can find other tips not only for indexing, but also for blog promotion and promotion in general in. Believe me, there are quite a lot of these chips, so tomorrow (if you suddenly start studying the course) you will be shocked at how many things you didn’t know!
But knowledge saves us from troubles when promoting a website. If so, read Webformyself, subscribe to us where it is convenient for you, and see you soon!
