huridocs/pdf-document-layout-analysis repository overview

⁠PDF Document Layout Analysis

A Docker-powered microservice for intelligent PDF document layout analysis, OCR, and content extraction

👁 Python Version
👁 FastAPI
👁 Docker
👁 GPU Support

⁠🚀 Overview

This project provides a powerful and flexible PDF analysis microservice built with Clean Architecture principles. The service enables OCR, segmentation, and classification of different parts of PDF pages, identifying elements such as texts, titles, pictures, tables, formulas, and more. Additionally, it determines the correct reading order of these identified elements and can convert PDFs to various formats including Markdown and HTML.

⁠✨ Key Features

• 🔍 Advanced PDF Layout Analysis - Segment and classify PDF content with high accuracy
• 🖼️ Visual & Fast Models - Choose between VGT (Vision Grid Transformer) for accuracy or LightGBM for speed
• 📝 Multi-format Output - Export to JSON, Markdown, HTML, and visualize PDF segmentations
• 🌐 OCR Support - 150+ language support with Tesseract OCR
• 📊 Table & Formula Extraction - Extract tables as HTML and formulas as LaTeX
• 🏗️ Clean Architecture - Modular, testable, and maintainable codebase
• 🐳 Docker-Ready - Easy deployment with GPU support
• ⚡ RESTful API - Comprehensive API with 10+ endpoints

⁠🔗 Project Links

• GitHub: pdf-document-layout-analysis⁠
• HuggingFace: pdf-document-layout-analysis⁠
• DockerHub: pdf-document-layout-analysis⁠

⁠Quick Start

Run the service:

• With GPU support:

docker run --rm --name pdf-document-layout-analysis --gpus '"device=0"' -p 5060:5060 --entrypoint ./start.sh huridocs/pdf-document-layout-analysis:v0.0.35

• Without GPU support:

docker run --rm --name pdf-document-layout-analysis -p 5060:5060 --entrypoint ./start.sh huridocs/pdf-document-layout-analysis:v0.0.35

📝 The service also has a user interface and a translation support but you should install it from the source. Check GitHub page for instructions.

Get the segments from a PDF:

curl -X POST -F 'file=@/PATH/TO/PDF/pdf_name.pdf' localhost:5060

To stop the server:

docker stop pdf-document-layout-analysis

⁠📋 Table of Contents

• 🚀 Quick Start⁠
• ⚙️ Dependencies⁠
• 📋 Requirements⁠
• 📚 API Reference⁠
• 💡 Usage Examples⁠
• 🏗️ Architecture⁠
• 🤖 Models⁠
• 📊 Data⁠
• 📈 Benchmarks⁠
  • Performance⁠
  • Speed⁠
• 🌐 Installation of More Languages for OCR⁠
• 🔗 Related Services⁠
• 🤝 Contributing⁠

⁠⚙️ Dependencies

⁠Required

• Docker Desktop 4.25.0+ - Installation Guide⁠
• Python 3.10+ (for local development)

⁠Optional

• NVIDIA Container Toolkit - Installation Guide⁠ (for GPU support)

⁠📋 Requirements

⁠System Requirements

• RAM: 2 GB minimum
• GPU Memory: 5 GB (optional, will fallback to CPU if unavailable)
• Disk Space: 10 GB for models and dependencies
• CPU: Multi-core recommended for better performance

⁠Docker Requirements

• Docker Engine 20.10+
• Docker Compose 2.0+

⁠📚 API Reference

The service provides a comprehensive RESTful API with the following endpoints:

⁠Core Analysis Endpoints

⁠Content Extraction Endpoints

⁠Format Conversion Endpoints

⁠OCR & Utility Endpoints

⁠Common Parameters

• file: PDF file to process (multipart/form-data)
• fast: Use LightGBM models instead of VGT (boolean, default: false)
• ocr_tables: Apply OCR to table regions (boolean, default: false)
• language: OCR language code (string, default: "en")
• types: Comma-separated content types to extract (string, default: "all")
• extract_toc: Include table of contents at the beginning of the output (boolean, default: false)
• dpi: Image resolution for conversion (integer, default: 120)

⁠💡 Usage Examples

⁠Basic PDF Analysis

Standard analysis with VGT model:

curl -X POST \
 -F '[email protected]' \
 http://localhost:5060

Fast analysis with LightGBM models:

curl -X POST \
 -F '[email protected]' \
 -F 'fast=true' \
 http://localhost:5060

Analysis with table OCR:

curl -X POST \
 -F '[email protected]' \
 -F 'ocr_tables=true' \
 http://localhost:5060

⁠Text Extraction

Extract all text:

curl -X POST \
 -F '[email protected]' \
 -F 'types=all' \
 http://localhost:5060/text

Extract specific content types:

curl -X POST \
 -F '[email protected]' \
 -F 'types=title,text,table' \
 http://localhost:5060/text

⁠Format Conversion

Convert to Markdown:

curl -X POST http://localhost:5060/markdown \
 -F '[email protected]' \
 -F 'extract_toc=true' \
 -F 'output_file=document.md' \
 --output 'document.zip'

Convert to HTML:

curl -X POST http://localhost:5060/html \
 -F '[email protected]' \
 -F 'extract_toc=true' \
 -F 'output_file=document.html' \
 --output 'document.zip'

⁠OCR Processing

OCR in English:

curl -X POST \
 -F 'file=@scanned_document.pdf' \
 -F 'language=en' \
 http://localhost:5060/ocr \
 --output ocr_processed.pdf

OCR in other languages:

# French
curl -X POST \
 -F 'file=@document_french.pdf' \
 -F 'language=fr' \
 http://localhost:5060/ocr \
 --output ocr_french.pdf

# Spanish
curl -X POST \
 -F 'file=@document_spanish.pdf' \
 -F 'language=es' \
 http://localhost:5060/ocr \
 --output ocr_spanish.pdf

⁠Visualization

Generate visualization PDF:

curl -X POST \
 -F '[email protected]' \
 http://localhost:5060/visualize \
 --output visualization.pdf

⁠Table of Contents Extraction

Extract structured TOC:

curl -X POST \
 -F '[email protected]' \
 http://localhost:5060/toc

⁠XML Storage and Retrieval

Analyze and save XML:

curl -X POST \
 -F '[email protected]' \
 http://localhost:5060/save_xml/my_analysis

Retrieve saved XML:

curl http://localhost:5060/get_xml/my_analysis.xml

⁠Service Information

Get service info and supported languages:

curl http://localhost:5060/info

Health check:

curl http://localhost:5060/

⁠Response Format

Most endpoints return JSON with segment information:

[
 {
 "left": 72.0,
 "top": 84.0,
 "width": 451.2,
 "height": 23.04,
 "page_number": 1,
 "page_width": 595.32,
 "page_height": 841.92,
 "text": "Document Title",
 "type": "Title"
 },
 {
 "left": 72.0,
 "top": 120.0,
 "width": 451.2,
 "height": 200.0,
 "page_number": 1,
 "page_width": 595.32,
 "page_height": 841.92,
 "text": "This is the main text content...",
 "type": "Text"
 }
]

⁠Supported Content Types

• Caption - Image and table captions
• Footnote - Footnote text
• Formula - Mathematical formulas
• List item - List items and bullet points
• Page footer - Footer content
• Page header - Header content
• Picture - Images and figures
• Section header - Section headings
• Table - Table content
• Text - Regular text paragraphs
• Title - Document and section titles

⁠🏗️ Architecture

This project follows Clean Architecture principles, ensuring separation of concerns, testability, and maintainability. The codebase is organized into distinct layers:

⁠Directory Structure

src/
├── domain/ # Enterprise Business Rules
│ ├── PdfImages.py # PDF image handling domain logic
│ ├── PdfSegment.py # PDF segment entity
│ ├── Prediction.py # ML prediction entity
│ └── SegmentBox.py # Core segment box entity
├── use_cases/ # Application Business Rules
│ ├── pdf_analysis/ # PDF analysis use case
│ ├── text_extraction/ # Text extraction use case
│ ├── toc_extraction/ # Table of contents extraction
│ ├── visualization/ # PDF visualization use case
│ ├── ocr/ # OCR processing use case
│ ├── markdown_conversion/ # Markdown conversion use case
│ └── html_conversion/ # HTML conversion use case
├── adapters/ # Interface Adapters
│ ├── infrastructure/ # External service adapters
│ ├── ml/ # Machine learning model adapters
│ ├── storage/ # File storage adapters
│ └── web/ # Web framework adapters
├── ports/ # Interface definitions
│ ├── services/ # Service interfaces
│ └── repositories/ # Repository interfaces
└── drivers/ # Frameworks & Drivers
 └── web/ # FastAPI application setup

⁠Layer Responsibilities

• Domain Layer: Contains core business entities and rules independent of external concerns
• Use Cases Layer: Orchestrates domain entities to fulfill specific application requirements
• Adapters Layer: Implements interfaces defined by inner layers and adapts external frameworks
• Drivers Layer: Contains frameworks, databases, and external agency configurations

⁠Key Benefits

• 🔄 Dependency Inversion: High-level modules don't depend on low-level modules

• 🧪 Testability: Easy to unit test business logic in isolation

• 🔧 Maintainability: Changes to external frameworks don't affect business rules

• 📈 Scalability: Easy to add new features without modifying existing code

⁠🤖 Models

The service offers two complementary model approaches, each optimized for different use cases:

⁠1. Vision Grid Transformer (VGT) - High Accuracy Model

Overview: A state-of-the-art visual model developed by Alibaba Research Group that "sees" the entire page layout.

Key Features:

• 🎯 High Accuracy: Best-in-class performance on document layout analysis
• 👁️ Visual Understanding: Analyzes the entire page context including spatial relationships
• 📊 Trained on DocLayNet: Uses the comprehensive DocLayNet dataset⁠
• 🔬 Research-Backed: Based on Advanced Literate Machinery⁠

Resource Requirements:

• GPU: 5GB+ VRAM (recommended)
• CPU: Falls back automatically if GPU unavailable
• Processing Speed: ~1.75 seconds/page (GPU [GTX 1070]) or ~13.5 seconds/page (CPU [i7-8700])

⁠2. LightGBM Models - Fast & Efficient

Overview: Lightweight ensemble of two specialized models using XML-based features from Poppler.

Key Features:

• ⚡ High Speed: ~0.42 seconds per page on CPU (i7-8700)
• 💾 Low Resource Usage: CPU-only, minimal memory footprint
• 🔄 Dual Model Approach:
  • Token Type Classifier: Identifies content types (title, text, table, etc.)
  • Segmentation Model: Determines proper content boundaries
• 📄 XML-Based: Uses Poppler's PDF-to-XML conversion for feature extraction

Trade-offs:

• Slightly lower accuracy compared to VGT
• No visual context understanding
• Excellent for batch processing and resource-constrained environments

⁠OCR Integration

Both models integrate seamlessly with OCR capabilities:

• Engine: Tesseract OCR⁠
• Processing: ocrmypdf⁠
• Languages: 150+ supported languages
• Output: Searchable PDFs with preserved layout

⁠Model Selection Guide

⁠📊 Data

⁠Training Dataset

Both model types are trained on the comprehensive DocLayNet dataset⁠, a large-scale document layout analysis dataset containing over 80,000 document pages.

⁠Document Categories

The models can identify and classify 11 distinct content types:

⁠Dataset Characteristics

• Domain Coverage: Academic papers, technical documents, reports
• Language: Primarily English with multilingual support
• Quality: High-quality annotations with bounding boxes and labels
• Diversity: Various document layouts, fonts, and formatting styles

For detailed information about the dataset, visit the DocLayNet repository⁠.

⁠📈 Benchmarks

⁠Performance

VGT model performance on PubLayNet dataset:

⁠Speed

Performance benchmarks on 15-page academic documents:

⁠Performance Recommendations

• GPU Available: Use VGT for best accuracy-speed balance
• CPU Only: Use LightGBM for optimal performance
• Batch Processing: LightGBM for consistent throughput
• High Accuracy: VGT with GPU for best results

⁠🌐 Installation of More Languages for OCR

The service uses Tesseract OCR with support for 150+ languages. The Docker image includes only common languages to minimize image size.

⁠Installing Additional Languages

⁠1. Access the Container

docker exec -it --user root pdf-document-layout-analysis /bin/bash

⁠2. Install Language Packs

# Install specific language
apt-get update
apt-get install tesseract-ocr-[LANGCODE]

⁠3. Common Language Examples

# Korean
apt-get install tesseract-ocr-kor

# German 
apt-get install tesseract-ocr-deu

# French
apt-get install tesseract-ocr-fra

# Spanish
apt-get install tesseract-ocr-spa

# Chinese Simplified
apt-get install tesseract-ocr-chi-sim

# Arabic
apt-get install tesseract-ocr-ara

# Japanese
apt-get install tesseract-ocr-jpn

⁠4. Verify Installation

curl http://localhost:5060/info

⁠Language Code Reference

Find Tesseract language codes in the ISO to Tesseract mapping⁠.

⁠Supported Languages

Common language codes:

• eng - English
• fra - French
• deu - German
• spa - Spanish
• ita - Italian
• por - Portuguese
• rus - Russian
• chi-sim - Chinese Simplified
• chi-tra - Chinese Traditional
• jpn - Japanese
• kor - Korean
• ara - Arabic
• hin - Hindi

⁠Usage with Multiple Languages

# OCR with specific language
curl -X POST \
 -F '[email protected]' \
 -F 'language=fr' \
 http://localhost:5060/ocr \
 --output french_ocr.pdf

⁠🔗 Related Services

Explore our ecosystem of PDF processing services built on this foundation:

⁠PDF Table of Contents Extractor⁠

🔍 Purpose: Intelligent extraction of structured table of contents from PDF documents

Key Features:

• Leverages layout analysis for accurate TOC identification
• Hierarchical structure recognition
• Multiple output formats supported
• Integration-ready API

⁠PDF Text Extraction⁠

📝 Purpose: Advanced text extraction with layout awareness

Key Features:

• Content-type aware extraction
• Preserves document structure
• Reading order optimization
• Clean text output with metadata

⁠Integration Benefits

These services work seamlessly together:

• Shared Analysis: Reuse layout analysis results across services
• Consistent Output: Standardized JSON format for easy integration
• Scalable Architecture: Deploy services independently or together
• Docker Ready: All services containerized for easy deployment

⁠🤝 Contributing

We welcome contributions to improve the PDF Document Layout Analysis service!

⁠How to Contribute

1. Fork the Repository

   git clone https://github.com/your-username/pdf-document-layout-analysis.git
   

2. Create a Feature Branch

   git checkout -b feature/your-feature-name
   

3. Set Up Development Environment

   make install_venv
   make install
   

4. Make Your Changes

   • Follow the Clean Architecture principles
   • Add tests for new features
   • Update documentation as needed

5. Run Tests and Quality Checks

   make test
   make check_format
   

6. Submit a Pull Request

   • Provide clear description of changes
   • Include test results
   • Reference any related issues

⁠Contribution Guidelines

⁠Code Standards

• Python: Follow PEP 8 with 125-character line length
• Architecture: Maintain Clean Architecture boundaries
• Testing: Include unit tests for new functionality
• Documentation: Update README and docstrings

⁠Areas for Contribution

• 🐛 Bug Fixes: Report and fix issues
• ✨ New Features: Add new endpoints or functionality
• 📚 Documentation: Improve guides and examples
• 🧪 Testing: Expand test coverage
• 🚀 Performance: Optimize processing speed
• 🌐 Internationalization: Add language support

⁠Development Workflow

1. Issue First: Create or comment on relevant issues
2. Small PRs: Keep pull requests focused and manageable
3. Clean Commits: Use descriptive commit messages
4. Documentation: Update relevant documentation
5. Testing: Ensure all tests pass

⁠Getting Help

• 📚 Documentation: Check this README and inline docs
• 💬 Issues: Search existing issues or create new ones
• 🔍 Code: Explore the codebase structure
• 📧 Contact: Reach out to maintainers for guidance

⁠License

This project is licensed under the terms specified in the LICENSE⁠ file.