Free Broken Link Checker & 404 Dead Link Analyzer

Broken Link Checker & Dead URL Auditor

An enterprise-grade, ultra-fast broken link checker, dead URL auditor, redirect chain tracker, and HTML5 anchor fragment verification engine for BrowserMesh.

Discovers, audits, classifies, and verifies hyperlinks, responsive picture sources (srcset), CDN stylesheets, scripts, videos, audio tracks, and embedded frames across entire websites or XML sitemaps using high-throughput HTTP checks and automatic Playwright stealth browser escalation.

The premier high-speed, free alternative to Screaming Frog (£209/yr), Ahrefs Broken Link Checker ($99+/mo), Sitebulb, and legacy single-threaded link checkers.

---

🎯 Target Audience & Industry Solutions

• Technical SEO Specialists & Web Auditors: Rapidly identify 404 Not Found and 410 Gone dead internal links draining Googlebot crawl budget, disrupting PageRank flow, and damaging organic search rankings.
• Website Redesign & Migration Engineers: Verify 301 and 308 redirect chains, eliminate cyclical redirect loops, detect canonical misconfigurations, and guarantee legacy URLs resolve seamlessly to live destination pages.
• Content Marketers & Technical Writers: Detect rotted external outbound links in blog articles, documentation centers, and knowledge bases with in-page anchor fragment verification (#section-id).
• E-Commerce Operations & Merchandisers: Ensure product imagery, gallery zoom slides, responsive <picture> descriptors, CDN stylesheets, and checkout iframe sources return HTTP 200 OK without broken media placeholders.
• CI/CD & DevOps Automation Engineers: Integrate automated link auditing into staging environments and pre-deployment pipelines to prevent pushing dead links or missing assets to production.

---

🚀 Key Features

1. 15+ HTML Tag & Asset Taxonomy

Audits hyperlinks and embedded media across all modern HTML5 publication vectors:

• Clickable Hyperlinks: <a href="..."> and <area href="..."> with rel="nofollow" detection and anchor text capture.
• Visual & Responsive Media: <img src="...">, <img srcset="...">, <source srcset="...">, <video src="..." poster="...">, <audio src="...">, and <track src="...">.
• Stylesheets & Scripts: <link rel="stylesheet" href="...">, <script src="...">, inline <style> url(...), and @import stylesheets.
• Embedded Documents: <iframe src="...">, <frame src="...">, and <object data="...">.
• Citations & Quotes: <blockquote cite="...">, <q cite="...">, <del cite="...">, and <ins cite="...">.
• Dynamic Redirects & Metadata: <meta http-equiv="refresh">, OpenGraph tags (og:url, og:image), Twitter cards (twitter:image), and JSON-LD structured data URLs (sameAs, image, logo).

2. Smart Dual-Engine Architecture (auto / http / browser)

• HTTP Mode: Pure native Node.js fetch() pipeline (~150–400ms per link check) with Chrome Desktop User-Agent, Brotli/Gzip decompression, and HEAD-with-GET fallback. Zero browser memory overhead.
• Browser Mode: Full Playwright stealth Chromium execution (CloakBrowser) for client-hydrated Single Page Applications (Next.js, React, Vue, SvelteKit) and JavaScript-rendered DOMs.
• Auto Mode (Default): Executes fast HTTP checks first; automatically escalates to stealth Playwright Chromium if a target host presents Cloudflare Turnstile, DataDome, ModSecurity WAF challenges, or client-rendered shells.

3. 4-Tier Noise Triaging (Eliminating 80%+ False Positives)

Traditional link checkers flood reports with false positives by marking Cloudflare 403 blocks or 429 rate limits as "broken". BrowserMesh strictly categorizes every check into 4 non-overlapping triage buckets:

1. ok: Verified live URL returning HTTP 200–299 or healthy redirect chain.
2. broken: Confirmed dead page (404 Not Found, 410 Gone, DNS ENOTFOUND, SPA Soft-404, or dead anchor fragment).
3. blocked: Live destination protected by anti-bot walls or authentication (401 Unauthorized, 403 Forbidden, 406 ModSecurity, 429 Rate Limit, 451 Legal, LinkedIn 999).
4. inconclusive: Transient network failure or upstream server outage (500 Internal Error, 502 Bad Gateway, 503 Unavailable, 504 Gateway Timeout, connection reset).

4. Recursive & Compressed XML Sitemap Ingestion (.xml & .xml.gz)

• Direct Sitemap Audits: Supply any sitemap URL (e.g. https://example.com/sitemap.xml) to audit all catalog URLs directly without manual page extraction.
• Sitemap Index Traversal: Automatically parses nested <sitemapindex> hierarchies down to all child partition files.
• Transparent Gzip Decompression: Seamlessly fetches and decompresses gzipped sitemaps (.xml.gz) commonly utilized by Shopify, Amazon S3, and large-scale publishers.

5. HTML5 In-Page Anchor Fragment Verification

Validates that targeted section hashes (https://example.com/docs#installation) actually exist on destination pages:

• Inspects target DOM for matching id="installation" or <a name="installation">.
• Supports HTML5 specification #top and #TOP exemptions.
• Handles GitHub-rendered markdown anchor prefixes (id="user-content-installation").
• Verifies W3C Scroll-to-Text directives (#:~:text=...).

6. Client-Side SPA Soft-404 Detection

Catches Single Page Applications (Next.js App Router, Vite, Remix, Angular) returning HTTP 200 OK while rendering "Page Not Found" templates by evaluating page <title>, primary headings (<h1>, <h2>), and meta robots noindex directives.

7. High-Performance Concurrency & Single-Flight Coalescing

• Single-Flight Promise Coalescing: When multiple concurrent workers encounter the same destination URL across different pages, they await the single active network request rather than making redundant calls.
• Shared Memory Cache: Each unique external URL is verified only once across an entire audit run, saving bandwidth and preventing rate-limit blocks.
• Polite Per-Host Jitter: Random 50–200ms delay prevents triggering target DDoS defenses.
• Retry-After Hierarchy: Complies with standard Retry-After and RateLimit-Reset headers on HTTP 429 responses.

8. Domain-Specific Bot Adaptation & Resilience

• YouTube oEmbed API Validation: Probes the official YouTube oEmbed API (https://www.youtube.com/oembed) rather than the heavy watch page, verifying video availability without bot blocks.
• Wikimedia Policy Compliance: Dispatches a dedicated descriptive User-Agent header for Wikipedia/Wikimedia links.
• Nginx HEAD Loop Recovery: Detects web servers that redirect HEAD requests to root in circles and automatically falls back to standard GET requests.
• Multi-Depth Queue Sanitization: Filters out static assets (.png, .css, .js) from crawl queues so only genuine HTML documents expand multi-depth traversals.

---

📊 Comparison with Alternatives

---

📋 Input Configuration

---

📊 Output Fields Reference (10 Fields)

Every audit yields detailed diagnostic records adhering to the BrowserMesh unified output specification:

---

💡 Best Practices for Large Enterprise Audits

1. Leverage XML Sitemaps: Pass https://example.com/sitemap.xml directly into urls. The engine automatically expands nested <sitemapindex> hierarchies down to all child partitions and decompresses .xml.gz archives.
2. Filter via onlyBroken for CI/CD: In automated deployment checks, enable onlyBroken: true to receive clean, compact reports containing only 404 dead links, broken images, and redirect anomalies.
3. Tune Concurrency Appropriately: Use concurrency: 10 for public internet audits to prevent tripping target DDoS rate limits; raise to concurrency: 20 for internal staging servers on high-bandwidth local networks.
4. Distinguish blocked from broken: Third-party sites (LinkedIn, Cloudflare-protected domains) will be classified under category: "blocked", ensuring your team focuses exclusively on genuine dead links rather than anti-bot false alarms.

---

❓ Frequently Asked Questions (FAQ)

Q: How does the plugin detect SPA soft-404 pages?

When an internal page returns HTTP 200 OK, the engine evaluates the <title> tag, primary headings (<h1>, <h2>), and meta robots noindex directives against a curated taxonomy of 404 patterns. If the content matches standard "Page Not Found" templates, the verdict is flagged as isBroken: true with category: "broken".

Q: Why are LinkedIn and Cloudflare links marked as "blocked" instead of "broken"?

LinkedIn returns HTTP 999 to automated scrapers, and Cloudflare frequently returns HTTP 403 or challenge pages. Because these links work perfectly for human visitors using standard web browsers, marking them as broken creates false positive noise. Triaging them as blocked keeps reports accurate.

Q: How are YouTube links verified without hitting bot blocks?

Rather than loading the heavy YouTube watch page which enforces strict bot throttling, the plugin queries the official YouTube oEmbed API (https://www.youtube.com/oembed?url=...). This returns HTTP 200 for live videos, 404 for deleted/private videos, and 401 for restricted content.

Q: Can I audit gzipped sitemaps (.xml.gz)?

Yes. The plugin includes a built-in decompression pipeline using native zlib.gunzipSync. It automatically detects gzip magic headers and file extensions, allowing seamless auditing of Shopify, Amazon S3, and enterprise sitemaps.

Q: How does the plugin handle Nginx redirect loops?

If a web server misconfigures HEAD requests to redirect in an infinite loop (e.g., returning HTTP 307 redirecting back to /), the engine automatically detects the cycle and recovers by executing a standard GET request.