Pricing

Rating

Developer

Actor stats

Categories

Share

🔎 Yahoo Search Scraper

Scrape Yahoo Search results at scale — titles, URLs, snippets, favicons, in-article sub-links, and a clean Markdown excerpt for every result. Bulk queries, time-window filtering and smart proxy auto-escalation (direct → datacenter → residential) keep your runs fast and unblocked.

⭐ Why Choose Us?

• Bulk-first — paste dozens of queries (or full Yahoo URLs) and walk every result page until the cap.
• Smart proxy ladder — starts direct, only escalates if Yahoo blocks. You don't pay for residential traffic you didn't need.
• Rich back-fill — when Yahoo's snippet is thin, the actor visits the result page and harvests in-article sub-links + a Markdown summary.
• Live results — rows stream to the dataset as they're scraped, so a mid-run interruption never loses your data.
• Production-grade error handling — 3-tier proxy retries, graceful PPE limit handling, exponential cool-downs.

🔑 Key Features

• 🌐 Bulk queries — plain keywords or full Yahoo SERP URLs, mixed freely.
• 📅 Time-window filter — Anytime / Past day / Past week / Past month.
• 🛡️ Auto-escalating proxy: direct → Apify Datacenter → Apify Residential (3 retries), then sticky.
• 🧩 Optional second-pass back-fill of sub-links + Markdown excerpts.
• 📋 Per-section dataset views: Overview, Snippet, Sub-links.
• 🔄 Custom proxy URLs supported — they go first, then the smart ladder.

🧾 Input

{
"queries":[
"java developer",
"https://search.yahoo.com/search?p=python+jobs"
],
"maxItems":10,
"timePeriod":"Anytime",
"backfillEmptyResults":true,
"backfillConcurrency":8,
"backfillMaxLinks":10,
"proxyConfiguration":{"useApifyProxy":false}
}

📤 Output

Each row matches the per-section views in the dataset.

{
"query":"java developer",
"title":"How to become a Java Developer? - GeeksforGeeks",
"url":"https://www.geeksforgeeks.org/gfg-academy/how-to-become-a-java-developer/",
"description":"A Java developer is a software engineer who builds...",
"text":" * Core Java\n\nCore Fundamentals: Learn concepts and practice DSA...\n",
"logo_url":"https://s.yimg.com/pv/.../32x32_7eae5aac8b7f7402.png",
"links":[
"https://www.geeksforgeeks.org/java/java",
"https://www.geeksforgeeks.org/advance-java/spring"
],
"domain":"www.geeksforgeeks.org"
}

🚀 How to Use (Apify Console)

1. Open Apify Console → Actors.
2. Find this actor and open it.
3. Paste your queries (one per line) into 🌐 Search Queries / URLs.
4. Pick a 🎁 Maximum results cap and a 📅 Time window.
5. (Optional) Leave proxy on direct — the actor will auto-escalate only when needed.
6. Click Start.
7. Watch live logs — rows appear in the Output tab as they're scraped.
8. Export results as JSON / CSV / XLSX.

🤖 Use via API

curl-X POST "https://api.apify.com/v2/acts/<ACTOR_ID>/run-sync-get-dataset-items?token=$APIFY_TOKEN"\
-H"Content-Type: application/json"\
-d'{
 "queries": ["java developer"],
 "maxItems": 10,
 "timePeriod": "Anytime"
 }'

💼 Best Use Cases

• SEO & SERP monitoring on Yahoo.
• Competitive intelligence — track who appears for a query over time.
• Lead generation — feed result URLs into your own enrichment pipeline.
• Content discovery — harvest in-article sub-links for further crawling.

💳 Pricing

This actor uses Apify's Pay-per-event model. The primary event is result-item — one charge per result row pushed to the dataset. You pay only for the rows you actually receive; back-fill, retries and failed attempts are not billed.

You also pay the underlying Apify platform usage (compute units, proxy traffic when used). Direct (no-proxy) requests cost no proxy traffic at all — which is why the actor stays on direct until Yahoo forces it to escalate.

❓ Frequently Asked Questions

Does it work when Yahoo blocks me? Yes. The default no-proxy run is the fastest, but the moment Yahoo returns a block (HTTP 429/503 or a captcha page), the actor auto-escalates to the Apify Datacenter pool, then to Residential with up to 3 retries. Once a tier works, it's locked in for the rest of the run.

Can I bring my own proxies? Yes — paste them into the proxy field's Custom proxy URLs. Your URLs are tried first (3 retries), then the datacenter → residential fallback ladder kicks in.

Does it follow pagination? Yes. Yahoo returns ~7 results per page; the actor walks pages until your maxItems cap is hit or 3 consecutive pages return nothing.

What about non-Latin queries? Yahoo handles UTF-8 queries natively — paste them as-is.

Why is my back-filled text empty for some rows? Some sites block all bots (or render with JS only). In that case the actor falls back to a minimal block built from Yahoo's own title + description so the field is never blank.

📨 Support & Feedback

• Issues / feature requests → please open a thread on the actor's detail page.
• Custom solutions → dev.scraperengine@gmail.com.