VOOZH about

URL: https://apify.com/crawlergang/data-gov-catalog-scraper

⇱ Data.gov Catalog Scraper Β· Apify


Pricing

from $3.00 / 1,000 results

Go to Apify Store

Data.gov Catalog Scraper

Scrape the Data.gov catalog (catalog.data.gov). Search 300,000+ open government datasets by keyword, organization, and format. Fetch dataset details or list organizations. No API key required.

Pricing

from $3.00 / 1,000 results

Rating

5.0

(11)

Developer

πŸ‘ Crawler Gang

Crawler Gang

Maintained by Community

Actor stats

11

Bookmarked

2

Total users

1

Monthly active users

8 days ago

Last modified

Share

NASA Open Data Scraper

Search and extract metadata from NASA's Open Data Portal β€” powered by CKAN with 40,000+ NASA datasets covering Earth science, space exploration, aeronautics, climate, and more. No API key required.

What you can scrape

  • Datasets β€” titles, descriptions, resources (files), tags, organization, license, metadata dates, resource counts
  • Dataset detail β€” full metadata for specific datasets by ID or slug
  • Organizations β€” NASA organizations publishing open data, with package counts

Use cases

  • NASA research data discovery and analysis
  • Finding Earth science, climate, or space datasets
  • Academic and scientific research
  • Building data pipelines from NASA open sources
  • Tracking dataset availability by NASA program
  • Discovering datasets in specific formats (CSV, GeoJSON, HDF, FITS, etc.)

Input

FieldTypeDescription
modeselectsearchDatasets (default), getDatasetDetail, or listOrganizations
searchQuerystringKeyword(s) to search dataset titles and descriptions
organizationstringFilter by NASA organization slug (e.g. nasa)
formatselectFilter by resource format: CSV, JSON, GeoJSON, ZIP, PDF, etc.
sortByselectSort by relevance, recently modified, recently added, or name
datasetIdsarrayDataset IDs or slugs for getDatasetDetail mode
maxItemsintegerMax records to return (1–5000, default 25)

Example inputs

Search climate datasets:

{
"mode":"searchDatasets",
"searchQuery":"climate",
"maxItems":50
}

Search Mars datasets in ZIP format:

{
"mode":"searchDatasets",
"searchQuery":"mars",
"format":"ZIP",
"maxItems":100
}

Get specific dataset details:

{
"mode":"getDatasetDetail",
"datasetIds":["tracking-climate-models"],
"maxItems":10
}

List all NASA organizations:

{
"mode":"listOrganizations",
"maxItems":200
}

Output

Dataset record

{
"datasetId":"296fce57-6171-425e-ae6d-ba0830e27a26",
"datasetSlug":"islscp-ii-climate-research-unit-cru05-monthly-climate-data-d1e5f",
"title":"ISLSCP II Climate Research Unit CRU05 Monthly Climate Data",
"description":"This data set contains a mean monthly climatology for several climate variables...",
"organizationName":"nasa",
"organizationTitle":"NASA",
"maintainer":"Earthdata Forum",
"maintainerEmail":"earthdata-support@nasa.gov",
"tags":["earth-science-atmospheric-temperature","earth-science-precipitation"],
"resources":[
{
"name":"Dataset Download",
"format":"ZIP",
"url":"https://data.ornldaac.earthdata.nasa.gov/...",
"created":"2025-08-21T18:27:40.671929"
}
],
"licenseTitle":"other-license-specified",
"isOpen":false,
"metadataCreated":"2025-04-01T16:43:52.479370",
"metadataModified":"2026-05-18T19:57:44.199238",
"numResources":7,
"numTags":5,
"sourceUrl":"https://data.nasa.gov/dataset/296fce57-6171-425e-ae6d-ba0830e27a26",
"scrapedAt":"2026-05-30T08:30:00+00:00",
"recordType":"dataset"
}

Organization record

{
"orgName":"nasa",
"orgTitle":"NASA",
"orgDescription":"NASA's vision: To reach for new heights and reveal the unknown...",
"memberCount":0,
"packageCount":40000,
"sourceUrl":"https://data.nasa.gov/organization/nasa",
"scrapedAt":"2026-05-30T08:30:00+00:00",
"recordType":"organization"
}

FAQ

Do I need an API key? No. NASA's Open Data Portal CKAN API is fully open and does not require authentication.

How many datasets are available? Over 40,000 NASA datasets covering Earth science, space exploration, aeronautics, climate research, and more.

Can I filter by file format? Yes. Use the format field to filter by CSV, JSON, ZIP, PDF, GeoJSON, and more.

Can I search by NASA organization? Yes. Use the organization field with nasa as the slug.

What's the difference between searchDatasets and getDatasetDetail? searchDatasets performs keyword search and returns many results. getDatasetDetail fetches complete metadata for specific datasets by their ID or slug.

Why are some fields missing from output? Only fields with actual data are included. No null or blank values are emitted.

Can I get datasets sorted by date? Yes. Set sortBy to metadata_modified desc for recently updated datasets or metadata_created desc for newest additions.

What science topics are covered? NASA's portal includes data from Earth science, space physics, planetary science, astrophysics, aeronautics, human spaceflight, and climate research programs.

You might also like

Data.gov Catalog Scraper

crawlerbros/data-gov-catalog-scraper

Scrape the Data.gov catalog (catalog.data.gov). Search 300,000+ open government datasets by keyword, organization, and format. Fetch dataset details or list organizations. No API key required.

Data.gov API - US Open Government Datasets

alizarin_refrigerator-owner/data-gov-api---us-open-government-datasets

Access the Data.gov catalog of 300,000+ US government datasets. Search datasets by topic, agency, format, and keywords. Discover open data from federal, state, and local governments

Data.gov.uk Scraper - Low-costπŸ’²πŸ”₯πŸ“šπŸ‡¬πŸ‡§

delectable_incubator/data-gov-uk-scraper-low-cost

Scrape data.gov.uk dataset listings πŸ”ŽπŸ“Š with a powerful open data scraper. Extract dataset titles, publishers, update dates, descriptions, tags, and dataset URLs from search results. Ideal for government data monitoring, open data research, dataset discovery, and structured data catalog creation πŸš€

Data.gov Dataset Catalog Crawler

jungle_synthesizer/datagov-dataset-crawler

Crawl 300K+ US government datasets from Data.gov. Extract titles, organizations, tags, formats, download URLs, API endpoints, temporal and spatial coverage, contacts, and resources. Filter by agency, format, category, and tags.

πŸ‘ User avatar

BowTiedRaccoon

2

Australia Open Data (data.gov.au) Scraper

parseforge/australia-data-gov-au-scraper

Export Australian government open datasets from data.gov.au. Browse 70k+ datasets across federal, state, and territory agencies. Pull dataset metadata, resources, organization, license, tags, and update frequency. Catalog mode lists all; dataset mode fetches one by ID.

Data.gov.uk Scraper - Cheap πŸŒπŸ“ŠπŸ‡¬πŸ‡§

scrapestorm/data-gov-uk-scraper---cheap

πŸ”Ž Easily collect dataset listings from data.gov.uk Provide one or multiple search URLs and extract dataset information such as πŸ“„ Dataset Title 🏒 Published By πŸ•’ Last Updated πŸ“ Description πŸ”— Dataset URL & more Perfect for open data research, government data monitoring & dataset discovery πŸ“ŠπŸš€

1

5.0

France Data.gouv.fr Scraper | French Open Data Catalog

parseforge/france-data-gouv-scraper

Export the French government open data catalog: dataset title, slug, description, organization, license, tags, created date, last modified, resources count and download URL. Search by keyword or filter by organization. CSV, Excel, JSON, XML for research and policy analysis.