Website visibility and page rules of the SEO plugin
Distinguish a physical robots.txt from a virtual WordPress output. Changes in the plugin do not necessarily access an existing file.
Compare the basic visibility setting of the installation with the specification for the individual contribution or archive. A setup adopted from the test phase can appear wider than a single noindex field.
Where to check the setting in WordPress
Under Settings → Read the search engine visibility and then the indexing rule of the active SEO plugin for the affected page type. For core outputs, the filter wp_robots is relevant. A robots.txt lock and a noindex tag are separate specifications.
Example of this specific finding
Illustrative control case: A rule for /offers/ also captures a published service page that should appear in the search.
follow-up audit in WordPress
Check the homepage, contribution and archive for the actually delivered meta and HTTP specifications. A global change does not derive from a single unintentionally excluded contribution.
Check the actually effective user-agent group and suitable path rule. Test with a permitted and a still blocked control URL.
Derogations and scope
An intentional crawl exclusion is not automatically an error. Confidential content requires additional access control.
Related: WordPress with classic or block theme. SEO plugins and page builders are different; there is no universal menu path for all installations.
Classify the general error
Definition, system-independent causes and further correction examples: Finding the Effective Rule for the Right Crawler
Configuration and test signal documentation
WordPress: wp_robots and filters