HTTP 404
The HTTP 404 error is perhaps the most famous and frustrating message on the internet, a digital dead-end we've all encountered. It signifies that while your browser successfully connected with a server, the specific content you asked for simply couldn't be found. But this humble four-digit code holds more stories and complexities than you might imagine, revealing hidden corners of web infrastructure and even digital censorship. A 404 error confirms your connection to a server but signals that the requested resource is missing or intentionally hidden. Not all 'page not found' messages are true 404s; some are 'soft 404s' that deceptively return an 'OK' status. Beyond a technical hiccup, the 404 has cultural significance and can even be used for creative expression or, more disturbingly, as a tool for censorship.
AI Summary
The HTTP 404 error is perhaps the most famous and frustrating message on the internet, a digital dead-end we've all encountered. It signifies that while your browser successfully connected with a server, the specific content you asked for simply couldn't be found. But this humble four-digit code holds more stories and complexities than you might imagine, revealing hidden corners of web infrastructure and even digital censorship.
- A 404 error confirms your connection to a server but signals that the requested resource is missing or intentionally hidden.
- Not all 'page not found' messages are true 404s; some are 'soft 404s' that deceptively return an 'OK' status.
- Beyond a technical hiccup, the 404 has cultural significance and can even be used for creative expression or, more disturbingly, as a tool for censorship.
The Enigmatic 404 Not Found
Every time you browse the web, your browser speaks a language called HTTP, or Hypertext Transfer Protocol. It sends requests to web servers, asking for pages, images, or videos. When everything goes smoothly, you see your content. But sometimes, you get a polite, yet firm, refusal.
That refusal often comes in the form of the HTTP 404 status code. It doesn't mean the server is offline or your internet is down. Instead, it's a specific message: 'I hear you loud and clear, but the item you're looking for just isn't here.' It's like asking a librarian for a book, and they confirm the library is open, but that specific book isn't on the shelf — or perhaps never was.
This 'Not Found' message is incredibly common. Links break, pages get moved, websites are redesigned, or content is simply deleted. Because it's so frequently encountered, the 404 has become one of the most recognizable and, at times, most frustrating symbols of the World Wide Web.
Deconstructing the Status Code
HTTP status codes are always three digits long, each carrying a specific meaning. The first digit categorizes the type of response. A '4' indicates a client error—meaning the problem lies with the request itself, perhaps a mistyped URL or a request for a non-existent resource. The '04' specifies the exact nature of that client error.
While 404 means 'Not Found,' other codes offer more specific guidance. For example, if a page has permanently moved, the server should ideally send a 301 'Moved Permanently' code, which tells search engines and browsers where to find the new location. If content is truly gone forever, a 410 'Gone' status is more appropriate.
It's also crucial not to confuse a 404 error with a DNS error. A DNS (Domain Name System) error means your browser couldn't even find the server itself—it's like the phone book doesn't list the address. A 404, however, means your browser found the server, but the server couldn't locate the specific page or file you requested.
The Deceptive 'Soft 404'
Sometimes, a website might display a 'Page Not Found' message, but technically, it's lying to your browser. This is called a 'soft 404.' The server actually sends a '200 OK' status code, pretending everything is fine, even though the content you asked for isn't there.
Soft 404s pose a significant problem for search engines and automated tools. These systems rely on accurate status codes to understand the web. If a server says 'OK' for a missing page, search engines might mistakenly index it as valid content, leading to more broken links in search results.
These deceptive responses often arise from server misconfigurations—for instance, an error page might be set up to always return an 'OK' status instead of the correct 404. In some cases, websites intentionally use soft 404s to force certain browsers to display a custom error page rather than a generic, browser-generated one.
A similar deception occurs with 'soft 3XX' errors. Here, missing pages might redirect to the website's homepage, again returning a '200 OK' status. While seemingly harmless, this can dilute the homepage's search engine authority and make it harder to identify broken internal links.
404s Beyond the Browser
Proxy Server Peculiarities
Even proxy servers can sometimes muddy the waters. If a proxy can't fulfill a request due to a problem with the remote server—like a hostname failing to resolve or a connection being refused—it should really send a 5xx 'Server Error' code. Yet, some proxies incorrectly send a 404, making it harder for programs to diagnose the actual issue.
When 404 Means Censorship
In a more unsettling application, the 404 error has been used as a tool for censorship. Instead of a '403 Forbidden' code, which clearly states access is denied, some internet service providers and governments return a 404 for blocked content. This subtly conceals the act of censorship by implying the content simply doesn't exist.
This practice was notably observed in Tunisia before the 2011 revolution. People became so familiar with these fake 404s that they created an imaginary character called 'Ammar 404' to personify the invisible censor, highlighting the subtle but pervasive nature of digital control.
Deep Dives: IIS Substatus Codes
While standard HTTP status codes are universal, some web server software offers additional layers of detail. Microsoft's Internet Information Services (IIS), for example, provides 'substatus codes' appended to the 404, like '404.5' or '404.14.' These are not officially recognized by the IANA but offer more specific clues about why a resource couldn't be found within an IIS environment.
These substatus codes can pinpoint very specific issues, such as '404.1 — Site Not Found,' '404.3 — MIME type restriction,' or '404.14 — Request URL too long.' They are essentially internal diagnostic messages to help administrators troubleshoot server configurations.
Customizing the Digital Dead End
Given how often users encounter 404s, web developers often configure custom error pages. These tailored pages can replace the generic browser message with a more user-friendly design, incorporating the website's branding, helpful navigation links, a search bar, or even a sitemap to guide users back to valid content.
However, browsers like Internet Explorer and Google Chrome have historically applied a minimum size rule. If a custom 404 page is too small, they'll often revert to displaying their own 'friendly' error message instead, trying to be helpful but sometimes overriding a website's carefully crafted design.
Beyond mere usability, custom 404 pages have become a canvas for creativity and social impact. Many organizations inject humor, display whimsical artwork, or even use them for political messaging. They transform a frustrating experience into an opportunity for engagement.
A powerful example is the 'NotFound project' in Europe, where organizations encourage websites to add a snippet of code to their 404 pages. This code displays information about missing children, turning a moment of frustration into a potential chance to help bring a child home.
Tracking Down the Missing Pages
For website owners, understanding where and why 404 errors occur is crucial for maintaining a healthy site and good user experience. Tools exist to crawl websites and find internal links that lead to 404s, helping webmasters fix their own broken connections.
However, a significant number of 404s come from external links—from other websites linking incorrectly to yours. More advanced tools like Google Search Console and Google Analytics are invaluable for tracking these external 404s, revealing how users are actually encountering 'Not Found' pages on your site.
Analyzing server log files or integrating JavaScript-based tracking directly onto custom 404 pages can also provide detailed insights. By understanding the common paths users take to these digital dead ends, website administrators can proactively improve navigation, fix broken links, and ensure content is always where it's expected to be.
Article
HTTP 404
English Wikipedia's 404 page
In HTTP, the 404 HTTP status code indicates that a web client (i.e. browser) was able to communicate with a server, but the server could not provide the requested resource. The server may not have the resource or it may not wish to disclose whether it has the resource. The code is often associated with response reason Not Found and is often referred to as page not found or file not found.
Often, the server generates a web page for the condition and the client displays it, and often the content indicates the error code. Since the condition is relatively commonly-encountered (i.e. due to broken or dead links), this code is one of the most recognizable on the World Wide Web.
Overview
HTTP 404
When communicating via HTTP, a server is required to respond to a request, such as a web browser request for a web page, with a numeric response code and an optional, mandatory, or disallowed (based upon the status code) message. In code 404, the first digit indicates a client error, such as a mistyped Uniform Resource Locator (URL). The following two digits indicate the specific error encountered. HTTP's use of three-digit codes is similar to the use of such codes in earlier protocols such as FTP and NNTP. At the HTTP level, a 404 response code is followed by a human-readable "reason phrase". The HTTP specification suggests the phrase "Not Found" and many web servers by default issue an HTML page that includes both the 404 code and the "Not Found" phrase.
A 404 error is often returned when pages have been moved or deleted. In the first case, it is better to employ URL mapping or URL redirection by returning a 301 Moved Permanently response, which can be configured in most server configuration files, or through URL rewriting; in the second case, a 410 Gone should be returned. Because these two options require special server configuration, most websites do not make use of them.
404 errors should not be confused with DNS errors, which appear when the given URL refers to a server name that does not exist. A 404 error indicates that the server itself was found, but that the server was not able to retrieve the requested page.
Soft 404 errors
HTTP 404
Some websites report a "not found" error by returning a standard web page with a "200 OK" response code, falsely reporting that the page loaded properly; this is known as a soft 404. The term "soft 404" was introduced in 2004 by Ziv Bar-Yossef et al.
Soft 404s are problematic for automated methods of discovering whether a link is broken. Some search engines, like Yahoo and Google, use automated processes to detect soft 404s. Soft 404s can occur as a result of configuration errors when using certain HTTP server software, for example with the Apache software, when an Error Document 404 (specified in a .htaccess file) is specified as an absolute path (e.g. http://example.com/error.html) rather than a relative path (/error.html). This can also be done on purpose to force some browsers (like Internet Explorer) to display a customized 404 error message rather than replacing what is served with a browser-specific "friendly" error message (in Internet Explorer, this behavior is triggered when a 404 is served and the received HTML is shorter than a certain length, and can be manually disabled by the user).
There are also "soft 3XX" errors where content is returned with a status 200 but comes from a redirected page, such as when missing pages are redirected to the domain root/home page.
Proxy servers
Some proxy servers generate a 404 error when a 500-range error code would be more correct. If the proxy server is unable to satisfy a request for a page because of a problem with the remote host (such as hostname resolution failures or refused TCP connections), this should be described as a 5xx Internal Server Error, but might deliver a 404 instead. This can confuse programs that expect and act on specific responses, as they can no longer easily distinguish between an absent web server and a missing web page on a web server that is present.
Intentional 404s
In July 2004, the UK telecom provider BT Group deployed the Cleanfeed content blocking system, which returns a 404 error to any request for content identified as potentially illegal by the Internet Watch Foundation. Other ISPs return a HTTP 403 "forbidden" error in the same circumstances. The practice of employing fake 404 errors as a means to conceal censorship has also been reported in Thailand and Tunisia. In Tunisia, where censorship was severe before the 2011 revolution, people became aware of the nature of the fake 404 errors and created an imaginary character named "Ammar 404" who represents "the invisible censor".
Microsoft Internet Server 404 substatus error codes
HTTP 404
The webserver software developed by Microsoft, Microsoft's Internet Information Services (IIS), returns a set of substatus codes with its 404 responses. The substatus codes take the form of decimal numbers appended to the 404 status code. The substatus codes are not officially recognized by IANA and are not returned by non-Microsoft servers.
Substatus codes
Microsoft's IIS 7.0, IIS 7.5, and IIS 8.0 servers define the following HTTP substatus codes to indicate a more specific cause of a 404 error:
• 404.0 – Not found. • 404.1 – Site Not Found. • 404.2 – ISAPI or CGI restriction. • 404.3 – MIME type restriction. • 404.4 – No handler configured. • 404.5 – Denied by request filtering configuration. • 404.6 – Verb denied. • 404.7 – File extension denied. • 404.8 – Hidden namespace. • 404.9 – File attribute hidden. • 404.10 – Request header too long. • 404.11 – Request contains double escape sequence. • 404.12 – Request contains high-bit characters. • 404.13 – Content length too large. • 404.14 – Request URL too long. • 404.15 – Query string too long. • 404.16 – DAV request sent to the static file handler. • 404.17 – Dynamic content mapped to the static file handler via a wildcard MIME mapping. • 404.18 – Query string sequence denied. • 404.19 – Denied by filtering rule. • 404.20 – Too Many URL Segments.
Custom error pages
HTTP 404
The Wikimedia 404 message
Web servers can typically be configured to display a customised 404 error page, including a more natural description, the parent site's branding, and sometimes a site map, a search form or 404-page widget. The protocol level phrase, which is hidden from the user, is rarely customized. Internet Explorer, however, will not display custom pages unless they are larger than 512 bytes, opting instead to display a "friendly" error page. Google Chrome included similar functionality, where the 404 is replaced with alternative suggestions generated by Google algorithms, if the page is under 512 bytes in size. Another problem is that if the page does not provide a favicon, and a separate custom 404-page exists, extra traffic and longer loading times will be generated on every page view.
Many organizations use 404 error pages as an opportunity to inject humor into what may otherwise be a serious website. For example, Metro UK shows a polar bear on a skateboard, and the web development agency Left Logic has a simple drawing program. During the 2015 UK general election campaign the main political parties all used their 404 pages to either take aim at political opponents or show relevant policies to potential supporters. In Europe, the NotFound project, created by multiple European organizations including Missing Children Europe and Child Focus, encourages site operators to add a snippet of code to serve customized 404 error pages which provide data about missing children.
While many websites send additional information in a 404 error message—such as a link to the homepage of a website or a search box—some also endeavor to find the correct web page the user wanted. Extensions are available for some content management systems (CMSs) to do this.
Tracking 404 errors
HTTP 404
A number of tools exist that crawl through a website to find pages that return 404 status codes. These tools can be helpful in finding links that exist within a particular website. The limitation of these tools is that they only find links within one particular website, and ignore 404s resulting from links on other websites. As a result, these tools miss out on 83% of the 404s on websites. One way around this is to find 404 errors by analyzing external links.
One of the most effective ways to discover 404 errors is by using Google Search Console, Google Analytics or crawling software.
Another common method is tracking traffic to 404 pages using log file analysis. This can be useful to understand more about what 404s users reached on the site. Another method of tracking traffic to 404 pages is using JavaScript-based traffic tracking tools.