Broken Url Monitor

Pricing

Rating

Developer

Actor stats

Categories

Share

Stateful sitemap-based URL monitoring Actor.

This Actor continuously checks all URLs listed in a sitemap and detects regressions and changes over time, not just one-off failures.

What it does

• Parses sitemap files and sitemap indexes (nested sitemaps supported)
• Validates all URLs using HTTP HEAD (automatic fallback to GET)
• Stores a baseline snapshot on first run
• Compares subsequent runs against the previous snapshot
• Reports only meaningful changes, classified by severity

Detected changes

• Broken URLs (4xx / 5xx)
• Status code changes
• Newly introduced redirects
• URL recoveries (error → OK)
• Large response time regressions

Why this Actor

Most link checkers:

• Run once
• Dump raw status codes
• Require manual comparison

This Actor:

• Is stateful
• Safe to run repeatedly or on a schedule
• Produces diff-based output
• Designed for monitoring, not crawling

Input

First run (baseline)

On the first execution:

• All URLs are checked
• Results are stored as a baseline
• No diffs are reported

Example output:

{
"baseline":true,
"summary":{
"total":84,
"ok":84,
"redirect":0,
"clientError":0,
"serverError":0
},
"message":"Baseline stored. Monitoring is now active."
}

Subsequent runs (monitoring)

On later executions:

• URLs are rechecked
• Results are compared against the previous snapshot
• Only changes are reported

Example output:

{
"baseline":false,
"summary":{
"total":84,
"ok":82,
"errors":2
},
"changes":{
"critical":[
{
"url":"https://example.com/page",
"previous":{"status":200},
"current":{"status":404}
}
],
"warning":[],
"info":[]
},
"unchangedCount":82
}

Severity levels

• critical
  • Status code regressions (e.g. 200 → 404)
• warning
  • Newly introduced redirects
  • Response time regressions (>4× increase)
• info
  • URL recoveries
  • Newly added URLs

Recommended usage

• Run on a schedule (daily or hourly)
• Use Apify webhooks or integrations for alerts
• One Actor run equals one monitoring snapshot

Notes

• Each sitemap is monitored independently
• State is stored automatically per sitemap
• Safe to rerun indefinitely
• No proxies required for most sites

Webhook & Alerts

This Actor is designed to be used with Apify webhooks.

Example: Alert on broken URLs

Trigger condition:

• Event: ACTOR.RUN.SUCCEEDED
• Payload filter (pseudo):

payload.OUTPUT.baseline ===false&&
payload.OUTPUT.changes.critical.length >0

Example: Slack alert message (JSON)

{
"text":":rotating_light: Broken URLs detected",
"attachments":[
{
"fields":[
{"title":"Sitemap","value":"{{sitemapUrl}}"},
{"title":"Critical changes","value":"{{criticalChanges}}"},
{"title":"Warnings","value":"{{warningChanges}}"}
]
}
]
}

Example: Email summary

Send when:

• baseline === false
• criticalChanges > 0 Include:
• sitemap URL
• number of broken URLs
• link to Actor run