Cloaking: why it is problematic and how to avoid risks
What cloaking means, why it can violate guidelines and what clean alternatives there are.
The strongest effect usually arises when the topic is not only theoretically understood, but directly checked on real URLs, page types and search intentions.
What cloaking means, why it can violate guidelines and what clean alternatives there are. This wiki page arranges the topic practically and shows, When priority arises, which mistakes are often and how you derive concrete measures from them.
Example: Classify different bot responses
A call with a normal browser user agent provides HTTP 200, a call with a Googlebot string provides a security check. Compare status, forwarding, and main content. Such a deviation can occur due to a WAF and is initially an access problem, not evidence of intentional cloaking.
A released user-agent string does not confirm a real Googlebot request. For diagnosis, server logs and verified bot accesses help. Cloaking refers to misleadingly different content for manipulation; legitimate device customization is not cloaking simply because HTML differs.
Modern Cloaking: Objectives
The more modern method #2 aims to offer an optimized page to human users and at the same time promote SEO. Search engines primarily process text content and index it. Therefore, there is a text-based HTML page for them, which has been structurally optimized and should ensure a good ranking. Human visitors, on the other hand, use their browser to access a page with the same URL, which has been optimized for their needs and contains, among other things, multimedia content (videos, images, graphics), the content of which search engine crawlers can hardly identify (unless by signatures under the content).
Due to the concealment tactics, it is not apparent at first glance why this page ranks so well. Incidentally, this also confuses human competition. The content and optimization methods of successful pages are often copied, but this is not of much use in this case, because the success is based on a technology that is invisible to humans. This in turn can even be the basis for tangible fraud: With pages manipulated in this way, fraudulent advertising can also be spread, which a web crawler can not recognize.
Two cloaking techniques
In principle, two techniques can be distinguished in this concealment tactic:
- a) User Agent Cloaking: Various contents are shown that depend on the user agent, i.e. whether a person or a web crawler accesses the page.
- b) IP-Cloaking or IP-Delivery: The content differs depending on the IP address from which it is accessed.
Reaction of search engine operators to cloaking
Since this technique is clearly part of the banned Black Hat SEO, the search engine operators take action against it and impose a penalty on such a page, which almost always leads to delisting from the index. This is not a more harmless violation such as keyword spam or an over-optimization with backlinks, which those responsible at Google & Co. interpret as an outgrowth of an over-motivated optimizer and only with a reduction in the SERPs penalties.
Rather, the targeted concealment of websites for the purpose of deception is a deliberate fraud that search engine operators do not let pass. For their part, they have developed methods to expose such pages. One procedure is obviously that the crawlers search through websites via neutral IP addresses and with normal browser identifier, behave technically like a human and then compare the contents of URLs with those that the purely technical web crawler sees.
Such and other methods, which are not published by the search engine operators, run quietly and are in turn based on concealment tactics. For example, the search engine operators do not completely name the names of their web crawlers so that they are not assigned to a specific search engine. However, the dubious optimizers also upgrade, because the professional IP-Cloaking can not be easily uncovered. Which methods exist in detail on both sides (criminal optimizers and search engine operators), only the participants themselves know. In the SEO scene, this is the subject of assumptions.