Wayback Machine Bulk Lookup

Pricing

Rating

Developer

Actor stats

Categories

Share

Look up Wayback Machine (archive.org) snapshots for any URL or list of URLs. Returns the full capture timeline, optional snapshot HTML-to-markdown content, and a live-vs-snapshot text diff. Built for OSINT analysts, journalists verifying sources, SEO teams recovering link-rot, and legal evidence collection.

What this actor does

For each input URL, the actor:

1. Queries the Wayback CDX API to retrieve the snapshot index in your specified date range and capture limit
2. Optionally fetches snapshot HTML for each capture and converts it to markdown (for reading or archiving)
3. Optionally fetches the current live URL and computes a line-level text diff against the most recent snapshot (to detect page changes)

Each output record contains the full snapshot timeline plus optional diff and content fields.

Input

Example input:

{
"urls":[
"https://example.com/news/2024-article",
"https://example.com/about"
],
"dateFrom":"2023-01-01",
"dateTo":"2024-12-31",
"captureLimit":50,
"diffWithLive":true
}

Output

One record per input URL.

Example output record:

{
"url":"https://example.com/news/2024-article",
"snapshotCount":14,
"firstCaptured":"2024-03-12T08:42:00Z",
"lastCaptured":"2026-04-29T22:11:00Z",
"captures":[
"{\"timestamp\":\"2026-04-29T22:11:00Z\",\"archiveUrl\":\"https://web.archive.org/web/20260429221100/https://example.com/news/2024-article\",\"status\":200,\"mimetype\":\"text/html\"}"
],
"diff":{"addedLines":12,"removedLines":3,"changedRatio":0.04},
"liveStatus":200,
"finalLiveUrl":"https://example.com/news/2024-article",
"status":"success",
"errorMsg":null
}

Dataset views

The actor produces two dataset views in the Apify console:

• Capture Timeline — url, snapshotCount, firstCaptured, lastCaptured, captures
• Live vs Snapshot Diff — url, liveStatus, diff, lastCaptured

Rate limits and performance

The actor respects Wayback Machine's rate limits:

• CDX API queries: ~10 requests/second (110ms minimum delay)
• Snapshot content fetches: ~1-2 requests/second (700ms minimum delay)

For large batches with fetchSnapshotContent=true, expect longer runtimes. The default timeout is 2 hours. Start with a small captureLimit (e.g. 10) to estimate runtime before running at full scale.

Use cases

• OSINT / research: Check whether a source URL existed, when it was captured, and how its content has changed
• Journalism: Verify archived versions of articles or government pages for fact-checking
• SEO / link-rot recovery: Find archived versions of dead inbound links and plan redirects or outreach
• Legal evidence: Retrieve timestamped snapshots of web pages for documentation
• Web archiving: Bulk-check coverage for a list of URLs before deeper archiving work