🧮 Taxonomy Math w/ FM

🏆 Website | 🖥️ Code | 📖 Paper

A high-quality mathematics dataset curated from web data using taxonomy-based filtering, containing 34 billion tokens of mathematical content.

🎯 Dataset Overview

This dataset is part of the Essential-Web project, which introduces a new paradigm for dataset curation using expressive metadata and simple semantic filters. Unlike traditional math datasets that require complex domain-specific pipelines, our approach leverages a 12-category taxonomy to efficiently identify and extract high-quality mathematical content.

🔬 Taxonomy Math w/ FM (34B tokens): Documents labeled as 51 - Mathematics in our taxonomy, with all 116M recalled documents then scored by the FineMath classifier and filtered to the top 34B tokens.

🏆 Performance

Our taxonomy-based approach achieves competitive results with significantly less curation effort:

Results show our EAI-Taxonomy datasets perform within 15% of SOTA on GSM8K while requiring minimal domain-specific tuning. On MATH and MMLU-Math, EAI-Taxonomy Math w/ FM performs within standard error of the leading FineMath 3+ dataset.

✨ Key Features

• 🎯 Direct Distribution Targeting: Leverage existing taxonomy labels to target math content from web-scale data without training custom high-recall classifiers
• 🚀 Rapid Curation: Skip the expensive classifier training phase and go straight to content selection
• 💰 Cost Effective: Avoid the need to train high-recall domain-specific classifiers for content discovery
• 🔍 Two-Stage Approach: Use taxonomy for recall, then apply existing quality classifiers for selection
• 🌐 Web-Scale: Access to math content identified across 23.6B web documents

🛠️ Curation Method

Our approach simplifies math dataset creation:

1. Traditional Method: Train high-recall classifiers → Run on billions of documents
2. Our Method: Query taxonomy metadata for 51 - Mathematics → Apply FineMath classifier to all recalled documents → Select top-scoring content

Dataset Schema Documentation

Overview

This dataset contains web-crawled text data with comprehensive metadata, quality signals, and taxonomic classifications. Each record represents a document extracted from web archives with detailed provenance tracking and quality assessment metrics.

Core Fields

EAI Taxonomy Classification

Comprehensive hierarchical classification system with primary and secondary labels - the most important feature of this dataset. The taxonomy is designed to provide detailed subject categorization, document type identification, content quality assessment, and extraction quality indicators.

How to Load the Dataset

This section provides examples of how to load the EssentialAI/eai-taxonomy-math-w-fm dataset using different Python libraries and frameworks.

Using Hugging Face Datasets (Standard Method)

The simplest way to load the dataset is using the Hugging Face datasets library:

from datasets import load_dataset

# Load the entire dataset
dataset = load_dataset("EssentialAI/eai-taxonomy-math-w-fm")

# View dataset structure
print(dataset)
print(f"Number of examples: {len(dataset['train'])}")

You can also load the dataset in streaming mode to avoid downloading the entire dataset at once:

from datasets import load_dataset

# Load in streaming mode
dataset = load_dataset("EssentialAI/eai-taxonomy-math-w-fm", streaming=True)
data_stream = dataset["train"]

# Iterate through examples
for example in data_stream.take(5):
 print(example)

Using PySpark

For large-scale distributed processing, you can load the dataset using PySpark with the pyspark_huggingface library:

# First install the required library:
# pip install pyspark_huggingface

import pyspark_huggingface
from pyspark.sql import SparkSession

# Initialize Spark session
spark = SparkSession.builder.appName("EAI-Taxonomy-Math").getOrCreate()

# Load the dataset using the "huggingface" data source
df = spark.read.format("huggingface").load("EssentialAI/eai-taxonomy-math-w-fm")

# Basic dataset exploration
print(f"Dataset shape: {df.count()} rows, {len(df.columns)} columns")
df.show(10)
df.printSchema()

# Load only specific columns for efficiency
df_subset = (
 spark.read.format("huggingface")
 .option("columns", '["column1", "column2"]') # Replace with actual column names
 .load("EssentialAI/eai-taxonomy-math-w-fm")
)

# Run SQL queries on the dataset
df.createOrReplaceTempView("eai_math_dataset")
result = spark.sql("""
 SELECT COUNT(*) as total_examples
 FROM eai_math_dataset
""")
result.show()

Using Daft

Daft provides a modern DataFrame library optimized for machine learning workloads. You can load the dataset directly from Hugging Face:

import daft

# Load the entire dataset
df = daft.read_parquet("hf://datasets/EssentialAI/eai-taxonomy-math-w-fm")

# Basic exploration
print("Dataset schema:")
df.schema()

print("First 5 rows:")
df.show(5)

If you need to access private datasets or use authentication:

import daft
from daft.io import IOConfig, HTTPConfig

io_config = IOConfig(http=HTTPConfig(bearer_token="your_token"))
df = daft.read_parquet("hf://datasets/EssentialAI/eai-taxonomy-math-w-fm", io_config=io_config)

Installation Requirements

Make sure you have the required libraries installed:

# For Hugging Face datasets
pip install datasets

# For PySpark with Hugging Face integration
pip install pyspark_huggingface

# For Daft
pip install daft

📜 License

Essential-Web-v1.0 contributions are made available under the ODC attribution license; however, users should also abide by the Common Crawl - Terms of Use. We do not alter the license of any of the underlying data.

🎓 Citation

If you use this dataset, please cite our EssentialWeb paper:

@misc{ai2025essentialwebv1024ttokens,
 title={Essential-Web v1.0: 24T tokens of organized web data}, 
 author={Essential AI and : and Andrew Hojel and Michael Pust and Tim Romanski and Yash Vanjani and Ritvik Kapila and Mohit Parmar and Adarsh Chaluvaraju and Alok Tripathy and Anil Thomas and Ashish Tanwer and Darsh J Shah and Ishaan Shah and Karl Stratos and Khoi Nguyen and Kurt Smith and Michael Callahan and Peter Rushton and Philip Monk and Platon Mazarakis and Saad Jamal and Saurabh Srivastava and Somanshu Singla and Ashish Vaswani},
 year={2025},
 eprint={2506.14111},
 archivePrefix={arXiv},
 primaryClass={cs.CL},
 url={https://arxiv.org/abs/2506.14111}, 
}

10 items • Updated Jun 18, 2025 • 10