The Monster of Website Optimization – The 404 Monster

Users often find notices such as “Error 404 – Site not found” or “The site you requested was not found” on the internet.

The 404 monster has struck, and shows sites that no longer exist. This can include URLs as well as individual elements such as images, videos, or PDF files that cannot be reached.

For websites with lots of 404 pages, there's a risk that the defective URLs will use up the crawl budget of search engine bots at the cost of more important pages. Irritated visitors will also leave your website more quickly as a result of this negative user experience. Therefore, make sure that you only link to functioning webpages to avoid 404 errors at all costs.

You should regularly check your website in the Google Search Console and remove errors so that you can react quickly when the 404 monster appears. When you choose the option "crawl errors" under the tab "crawl" and enter your domain, you will learn which internal and external links are defective.

The 404 monster strikes, for example, when:

Canonical tags have been placed incorrectly, and thus incorrect sites have spread through the world wide web.
Contents that refer to a link were removed from the site architecture or were moved to another URL.
A file name changed, because, for example, you wanted to generate a more expressive URL
After relaunch, you didn't move the site. An invalid URL was requested; thus a URL that does not exist was entered into the address bar.

Our Monster Practice Example from E-Commerce

Often, online retailers have a problem when they deal in seasonal products, but do not remove the sites afterwards. The invalid URL is nevertheless visible in the SERPs. Searching for "lego batman 2017," a snippet from "videoeta.com" appears.

Figure 1: Snippet for an incorrect site

If you click on the link
http://videoeta.com/movie/awards.html?id=173676
the Googlebot gives the incorrect site. The reader, of course, goes back to the navigation but will not find any information relevant to the topic.

Figure 2: Individual 404 site

Find 404 Errors with Ryte

With Ryte, you can quickly identify 404 error pages and deactivate them. In the module Website Success, click on the menu point "Indexability" and then on "Status Codes." If the number on the 4xx (not found) bar is higher than zero, there is a need for action. If you click on the 4xx bar, all incorrect URLs are listed in the report underneath – in this example there are 6.

Figure 3: Status codes report in Ryte's module Website Success

In some cases, it makes sense to redirect 404 pages permanently using 301 redirect, for example, if there are pages that offer similar information to users, or to inherit link juice. However, lengthy redirects can cause new errors, and the page speed can be negatively influenced. You should always avoid internal 404 links, and you should correct incorrect links as quickly as possible. However, if you find out from your server logs or Google Analytics that a 404 page is often called up even though it is not internally linked, a backlink may be incorrect. You can either redirect this URL via 301-Redirect to a thematically appropriate, correct URL or find out from which website the backlink comes from, and ask the owner of this website to link to the correct URL.

Conclusion

See to it that your internal links are not generating any 404 errors, because otherwise you are giving away valuable link juice and are creating a poor user experience. If, however, 404 error pages remain because their contents have been permanently removed, they should contain the following elements:

The reason for the error,
Information with added value, and
Navigation possibilities.