Glossary of terms

The vocabulary around broken links and crawling is full of terms that sound interchangeable and are not. The definitions below are the ones used throughout this site, written in plain language and ordered alphabetically.

A to C

Alternative text

The text carried by the alt attribute of an image. It is displayed when the image cannot be loaded, announced by screen readers, and used by search engines to understand what the image shows. An image without alternative text becomes a blank gap when it breaks.

Anchor text

The visible, clickable words of a link. Descriptive anchor text tells both the visitor and the search engine what to expect at the destination, which is why "read our guide on fixing 404 errors" is more useful than "click here".

Broken link

A link whose destination does not answer correctly, whether because the page was deleted, the address contains a mistake, or the server is failing. An internal broken link points at your own site and is yours to repair. An external broken link points at another site and can break without any action on your part.

Canonical URL

The address you declare as the official one when the same content is reachable through several addresses, using a link rel="canonical" tag. It prevents search engines from treating variants such as the version with and without www, or with and without a trailing slash, as separate competing pages.

Crawl budget

The amount of crawling a search engine is willing to spend on your site in a given period. It is finite. Every request spent on a broken link, a redirect chain or a duplicate address is a request not spent discovering your real content. This is the main mechanism by which broken links indirectly harm a large site. Google explains how it is calculated in its guide to crawl budget management.

Crawler

A program that reads a page, extracts the links it contains, and repeats the operation on each of them. Search engines use crawlers to discover content. Link checkers use the same technique to map a site and test each address. Also called a spider or a robot.

D to L

Deep link

A link that points to a specific page inside a site rather than to its homepage. Deep links are more useful to visitors and carry more precise meaning for search engines, and they are also the ones most likely to break when a site is restructured.

Hotlinking

Displaying an image hosted on someone else's server directly in your own pages. The other site pays the bandwidth, which is why many block the practice, and your page loses the image the moment they do. Host your images on your own server.

HTTP status code

The three digit number a server returns with every response, indicating whether the request succeeded, was redirected, or failed. The full list is on our status code reference page.

Index

The database in which a search engine stores the pages it has crawled and judged worth keeping. A page that is crawled is not necessarily indexed, and a page that returns 404 is eventually removed from the index.

Internal linking

The network of links between the pages of your own site. It determines what a crawler can reach, how ranking signals are distributed across your pages, and how easily a visitor moves from one subject to a related one. Broken internal links tear holes in that network.

M to R

Mixed content

A page served over HTTPS that requests a resource, typically an image or a script, over plain HTTP. Browsers block the insecure request, so the resource silently fails to appear even though the file exists and the address is correct.

Orphan page

A page that exists and is reachable by anyone who knows its address, but that no other page links to. Crawlers cannot discover it, which means it will not appear in a link checker report and is unlikely to be indexed. Listing it in a sitemap helps, but a real internal link helps more.

Redirect

An instruction telling the browser that the requested resource lives at a different address. A permanent redirect, code 301 or 308, tells search engines to transfer the ranking signals and forget the old address. A temporary redirect, code 302 or 307, tells them to keep the old address indexed. Using the wrong one for a permanent move is a costly and common mistake.

Redirect chain

A sequence in which one redirect leads to another before reaching the final page. Every step costs a round trip, and crawlers may stop following before the end. Whenever you add a redirect, update any older rule that pointed at the address you just moved, so that the first request lands directly on the final destination. Our redirect checker finds them for you.

Redirect loop

Two or more redirects pointing at each other, so the request never arrives. The browser gives up after a few attempts and displays an error. Usually the result of an edit to a rewrite rule that did not account for an existing rule.

Robots.txt

A text file at the root of a site that tells crawlers which parts they may request. It controls crawling, not indexing: a page blocked in robots.txt can still appear in search results if other sites link to it, because the block prevents the crawler from reading the page but not from knowing it exists. The format was standardised by the IETF in RFC 9309, Robots Exclusion Protocol.

S to Z

Sitemap

An XML file listing the addresses you want search engines to know about, in the format defined by the sitemaps.org protocol. It helps them discover pages that internal linking reaches poorly, but it does not replace internal links and it does not guarantee indexing. A sitemap containing addresses that return 404 sends a poor signal, so keep it in step with the site.

Soft 404

A page that tells the visitor the content is missing while returning a 200 OK status code to the machine. Search engines index it because the server declared it valid, filling the index with copies of an error message. The repair is to send the real 404 header from the error page. Covered in detail on our broken images and soft 404 page.

Status code

See HTTP status code.

Timeout

The moment a client stops waiting for a server that has not answered. A link checker reporting timeouts on your pages is telling you something useful even though the pages are not strictly broken: real visitors are waiting just as long, and many of them do not.

Trailing slash

The slash at the end of an address. Depending on server configuration, /section and /section/ may be the same page, two pages, or one redirecting to the other. Choose one form, use it consistently in your links, and declare it in your canonical tags.

User agent

The identifier a client sends to say what it is: a browser, a search engine crawler, or a link checker. Servers sometimes treat these differently, which is why an automated tool can receive a 403 for a page that opens normally in a browser.

For the practical side of these terms, see our guide on fixing 404 errors and the frequently asked questions.