Link Decay on Reference Pages: Seven Failure Modes and the Checks That Catch Them

Reference pages carry a specific liability. A municipal energy guide, a university cost handbook, a campus security syllabus, or a curated resource list exists in order to point elsewhere, and the value of the page equals the value of its destinations. When a destination disappears, the page keeps its layout, its heading structure, and its confident anchor text while delivering nothing to the reader who follows the link.

Direct observation of reference pages and their outbound targets produced seven distinct failure modes. Several of them defeat the verification method most maintainers apply, which is a request against the linked domain rather than the linked route. Each finding below is stated with the identifiers observed at the time of testing.

A federal agency removed an entire section

energy.gov/energysaver/* returns 404. The condition was confirmed on five routes, among them /energysaver/air-sealing-your-home and /energysaver/energy-saver. The home page of energy.gov responds 200.

The shape of that result is the problem. A verifier that resolves and requests only the registered domain records energy.gov as healthy, reports zero issues, and leaves every link into the Energy Saver section broken. A second property compounds the miss: the error pages retain the generic <title> "Department of Energy". Verifiers that compare titles, or that accept a plausible institutional title as evidence of a live document, classify these 404 responses as live pages. Two independent heuristics fail on the same set of URLs, which is why removals of this kind persist in reference pages for long periods.

A national laboratory left DNS

nrel.gov, www.nrel.gov, pvwatts.nrel.gov, and rredc.nrel.gov return NOERROR with no A record. The result was reproduced against three independent public resolvers: 8.8.8.8, 1.1.1.1, and 9.9.9.9.

NOERROR without an A record differs from NXDOMAIN in a way that matters operationally. The domain exists in the .gov zone; no name under it carries an address. Tools that test for NXDOMAIN alone, or that treat any non-error DNS response as success, report the domain as present. A control query confirms the failure is specific rather than an artifact of the test environment: afdc.energy.gov, another subdomain operated by the same department, resolves normally.

The consequence for reference pages is broad. PVWatts served as the standard solar calculation reference cited by municipal and university pages, so every citation of that host now terminates in a resolution failure rather than an HTTP status a checker can read.

Anchor text outlives its destination

The energy efficiency page of the City of Bainbridge Island, Washington, carries a link with the anchor text "Caulking and weatherstripping" pointing to energy.gov/energysaver/air-sealing-your-home, which returns 404.

The anchor text remains an accurate description of a subject that no longer has a destination. Human review of the page reads as correct, because the visible words are correct. The defect is invisible without following the link. This failure mode explains why editorial review alone does not surface link decay: the signal a reader uses to judge relevance survives the loss of the resource.

An expired domain acquired a new owner

The construction cost guide published by the University of Missouri lists an entry labelled "Get-A-Quote" pointing to get-a-quote.net. The domain changed hands. It now serves an online casino in Russian, with the page <title> "Пинко Казино". Confirmed by direct access on 17 August 2026.

This case returns HTTP 200. Every status-code check passes. Every DNS check passes. The link is functionally worse than a 404, because an institutional page now routes readers to unrelated commercial content under an anchor that implies a vetted construction resource. Detection requires comparing the retrieved page against the expectation encoded in the anchor text or the surrounding context, which no status-based tool performs.

A publishing platform disappeared as a block

statistics.about.com, math.about.com, psychology.about.com, sociology.about.com, and canadaonline.about.com do not resolve. Educational pages that cited these hosts lost their references in a single event rather than one at a time.

Platform-level shutdowns produce correlated failures. A page citing four About.com subdomains lost four links simultaneously, which means the decay rate observed on a page is not a smooth function of age. Maintenance schedules built on an assumption of gradual attrition underestimate the damage a single platform retirement causes.

Crawler identity changes the response

app.dimensions.ai returns 404 to an identified crawler and 202 to a browser user-agent. In one batch of 26 URLs, 6 were false positives attributable to this behaviour.

That is roughly a quarter of the batch. Any dead-link survey that does not recheck failures with a browser user-agent overstates the problem, and the overstatement is large enough to invalidate a report. The corollary is that reported decay figures produced by unattended crawlers should be treated as upper bounds until each failure has survived a second request under a different identity.

Template links multiply the count

owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF) returns 404 because OWASP migrated away from MediaWiki. The same URL appeared in six distinct guides on a single campus, because it sat in a shared page element.

A naive count records six problems. One URL requires one decision and one edit. Counting by occurrence rather than by unique URL inflates the reported severity by a factor equal to the template's reach, and it misdirects remediation effort toward pages instead of toward the shared component.

Failure modes and detection

Method

Each URL was requested directly over HTTP and the returned status code recorded. Path-level requests were issued individually; no path status was inferred from any other path or from the domain root. Response titles were inspected separately from status codes so that the two signals could be compared rather than conflated.

DNS behaviour was tested by querying A records against 8.8.8.8, 1.1.1.1, and 9.9.9.9, and by distinguishing NXDOMAIN from NOERROR with an empty answer section. A control host under the same parent organisation, afdc.energy.gov, was queried to confirm that a negative result reflected the target rather than the test path.

Every failure was rechecked with a browser user-agent before classification. The 26-URL batch that produced 6 false positives established that this step is mandatory rather than optional. URLs were normalised and deduplicated before any count was produced, which reduced the six OWASP occurrences to a single unique defect.

Limitations

The observations record a single point in time. No longitudinal series exists here, so no decay rate per year can be derived from this data, and none is asserted. The total number of reference pages examined and the total number of outbound links inspected are not reported here.

The sample was not drawn at random. Pages were selected because they are reference-heavy, which biases the set toward domains with many outbound institutional citations. Generalising the proportions observed in the 26-URL batch to any larger population is unsupported.

Removal dates are unknown. The dates on which the Energy Saver section was withdrawn, the NREL zone lost its A records, and get-a-quote.net changed hands are not established here. Whether the NREL condition is permanent or a transient operational state cannot be determined from three resolver queries at one moment.

The content served at get-a-quote.net may change again. The finding records what that host returned on 17 August 2026 and carries no claim about its state before or after.

Conclusion

Maintenance of reference pages requires verification by route rather than by domain, because a 200 at the root says nothing about a 404 five path segments down. It requires rechecking every failure with a browser user-agent, because a single batch produced a 6-in-26 false positive rate without that step. It requires separating NXDOMAIN from NOERROR with no A record, because the second condition is a live zone with no reachable host and passes tests written for the first. It requires deduplication by URL before counting, because one bad link in a shared template presents as six. And it requires at least one content-level comparison, because an expired domain under new ownership returns 200 while serving a casino.