Book Metadata Scraper

Pricing

Rating

Developer

Actor stats

Categories

Share

Overview

The Book Metadata Scraper is an Apify Actor that extracts rich book metadata from the Open Library database. It accepts book titles or search queries and retrieves detailed information including author, ISBN, publisher, description, ratings, and cover images. Whether you're building a book database, conducting publishing research, or powering a recommendation engine, this actor delivers accurate and structured book metadata efficiently.

With proxy support and respectful rate limiting, it ensures reliable access to Open Library's public API without interruptions.

Features

• Multi-query Search – Search for multiple book titles or keywords in a single run.
• Deep Metadata Fetching – Retrieves extended details (description, work data) from the Open Library Works API when search results are incomplete.
• Cover Image Links – Returns direct URLs to book cover images via Open Library's cover service.
• ISBN Extraction – Prioritizes 13-digit ISBNs for each book result.
• Rating & Review Data – Includes average ratings and total ratings count where available.
• Proxy Support – Optionally uses Apify residential proxies to avoid IP blocking.
• Rate-Limit Friendly – Adds random delays between requests to respect API limits.
• Dataset Integration – Automatically pushes all book metadata to your Apify dataset for easy export.

How It Works

1. Input – Provide a list of book titles or search queries (e.g., "Atomic Habits", "Stephen King").
2. Search – The actor queries the Open Library search API (/search.json) for each query.
3. Deep Fetch – For each result, it fetches the Work detail page (/works/{key}.json) to retrieve missing fields like description.
4. Build Output – Structures all available metadata into a clean record and pushes it to the dataset.
5. Repeat – Processes all queries with a random delay between requests.

Input

Example input:

{
"queries":["Atomic Habits","The Alchemist","Stephen King"],
"max_results":5,
"proxyConfiguration":{
"proxyGroups":["RESIDENTIAL"],
"apifyProxyCountry":"US"
}
}

Output

Each book is pushed as a separate dataset record with the following fields:

Example output:

{
"title":"Atomic Habits",
"author":"James Clear",
"isbn":"9780735211292",
"publisher":"Avery",
"published_date":"2018",
"language":"ENG",
"pages":320,
"categories":["Self-Help","Habits","Psychology","Productivity","Nonfiction"],
"description":"No matter your goals, Atomic Habits offers a proven framework for improving every day...",
"average_rating":4.4,
"ratings_count":28000,
"price":"",
"currency":"USD",
"availability":"In Stock",
"cover_image":"https://covers.openlibrary.org/b/id/10527843-L.jpg",
"preview_link":"https://openlibrary.org/works/OL17930368W",
"source":"Open Library"
}

Use Cases

• Book Databases – Build and maintain a structured catalog of books and metadata.
• Recommendation Engines – Power book recommendation systems with rich metadata.
• Publishing Research – Analyze publishing trends, authors, and categories.
• E-commerce – Enrich product listings with book descriptions, covers, and ISBNs.
• Academic Research – Collect structured book data for literature or data science projects.
• Content Aggregation – Aggregate book information for blogs, apps, or reading platforms.

Quick Start

1. Open on Apify – Visit the actor page and click Try for free.
2. Set Input – Enter book titles or search keywords in the queries field.
3. Adjust Settings – Set max_results and optional proxy configuration.
4. Run the Actor – Start the run and monitor progress in the logs.
5. Download Results – Export the dataset as JSON, CSV, or Excel once finished.

Technical Stack

• Data Source – Open Library API (free, public)
• HTTP Client – requests with custom headers and optional proxy support
• Proxy – Apify Proxy (residential or datacenter)
• Platform – Apify Actor — serverless, scalable, integrated with Dataset and Key-Value Store

Related Tools

Changelog

v1.0.0 – Initial Release

• Multi-query search via Open Library API
• Deep metadata fetching from Works endpoint
• ISBN-13 prioritization
• Cover image and preview link generation
• Proxy configuration support
• Dataset integration with random request delays

Pricing

• Free for basic usage on Apify (up to certain compute limits).
• Paid plans available for higher volume, priority support, and longer runs.
• Proxy credits consumed if residential proxies are enabled.

Support & Feedback

• Issues & Ideas – Open a ticket on the Apify Actor issue tracker.
• Documentation – Visit Apify Docs for platform guides.
• API Notes – This actor uses the Open Library public API. Please use responsibly and avoid excessive request rates.

Disclaimer: This actor accesses publicly available data from Open Library. Please ensure your usage complies with Open Library's terms of service. This actor is intended for research and informational purposes only.