Technical SEO

Technical SEO has the longest checklists in the industry. It also has the shortest list of things that reliably change what ranks, and the gap between those two facts is where most DIY effort disappears.

This module is the short list. Four things account for nearly every technical problem I have been called in to diagnose, and they are all free to check.

robots.txt does not do what most people think

Start here, because the misunderstanding is common and expensive. robots.txt controls crawling, not indexing. Google states it directly: it “is not a mechanism for keeping a web page out of Google,” and “a page that’s disallowed in robots.txt can still be indexed if linked to from other sites.”

When that happens you get the worst of both. The URL appears in results, with no description, because Google was not allowed to crawl it and read one. To actually keep a page out, Google says to “block indexing with noindex or password-protect the page.”

And the corollary that catches people: a page carrying noindex that is also blocked in robots.txt will never be de-indexed, because Google cannot crawl it to see thenoindex. The two directives cancel each other out. If you want something gone, allow the crawl and let Google read the instruction.

Canonicals are a hint, not a command

Canonical tags tell Google which version of similar pages you consider the real one. They are advisory. Google’s documentation describes rel="canonical" as “a strong signal that the specified URL should become canonical” and notes that if you do not specify one, “Google will identify which version of the URL is objectively the best version to show to users in Search.”

Which explains the “Duplicate, Google chose different canonical than user” status in Search Console. That is not a bug. It is Google disagreeing with you, and the useful response is to work out why the two pages look the same to it rather than to re-issue the same instruction louder.

Worth saying plainly, because it causes real anxiety: there is no duplicate content penalty. Google’s canonicalization documentation does not mention one. What happens is consolidation, not punishment. Two near-identical pages do not get you fined, they get you one page ranking instead of two, which is a dilution problem and an architecture problem.

Have someone else look at the indexing problem

Redirects: permanent means something specific

Redirects are the highest-risk thing in this module, because a mistake is silent and permanent-ish. Google’s redirect documentation draws the distinction cleanly. A permanent redirect means “the indexing pipeline uses the redirect as a signal that the redirect target should be canonical,” and Google will “show the new redirect target in search results.” A temporary redirect “doesn’t use the redirect as a signal” and Google keeps showing the source page.

So a 302 on a permanent move tells Google to keep the old URL, which is the opposite of what you meant. Use 301 when the move is permanent, and check the status code rather than trusting the plugin.

Two habits worth having. Redirect to the closest equivalent page, never in bulk to the homepage, because a batch of unrelated URLs pointing at your homepage reads as a soft 404 and you lose the signal without transferring it. And collapse chains: if A goes to B goes to C, point A at C.

The JavaScript problem

If your site renders content in the browser rather than on the server, everything above can be correct and the page can still be effectively empty to a crawler. Google does render JavaScript, but it is a separate, later, less reliable stage than crawling the HTML.

There is a specific version of this worth knowing about, because it is invisible from the front end: Google can only follow a link if it “is an <a> HTML element with an href attribute.” A navigation built from click handlers works perfectly for every human visitor and does not exist as a path through your site.

The check is thirty seconds with the simulator below. The fix, if there is a problem, is a development change, and that is the third item in the cannot-DIY list.

What I would do in your position

One afternoon, four checks, in this order. Fetch robots.txt and read it. Search your source for noindex. Inspect three money pages in Search Console. Confirm one version of your domain redirects to the other.

If all four come back clean, you almost certainly do not have a technical problem, and the reason things are not ranking is content or links, which are module 6 and module 11. That is a genuinely useful thing to rule out, and most people never do because they are three weekends into a 200-point checklist instead.