The MCP WebAnalyzer server enables intelligent web content analysis and processing with AI-powered capabilities:

• Content Extraction & Conversion - Extract web pages to clean, structured markdown while removing ads, navigation, and clutter • AI-Powered Q&A - Answer questions about web content using OpenAI GPT models (GPT-3.5, GPT-4, GPT-4 Turbo, GPT-5) • Web Crawling - Discover subpages from URLs with configurable depth and page limits • Content Summarization - Generate concise one-line summaries and RAG-optimized extracts with optional question focusing • Smart Content Ranking - Analyze and rank content importance using custom algorithms • IDE Integration - Works with Claude Desktop, Cursor, VS Code, and PyCharm

Uses Celery for distributed asynchronous task processing and background job management

Supports containerized deployment and orchestration of the web analyzer services

Uses environment variable configuration for server settings, security keys, and connection information

Built on FastAPI to provide high-performance web analysis endpoints and API documentation

Integration with OpenAI's API for AI-powered web content analysis and summarization

Exposes metrics endpoints for monitoring server performance and usage statistics

Uses Redis for caching, session management, and as a message broker for task processing

1. Click on "Install Server".

2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.

3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@MCP WebAnalyzer summarize the key points from https://example.com/blog/post"

That's it! The server will respond to your query, and you can continue using it as needed.

Here is a step-by-step guide with screenshots.

# Clone the repository
git clone https://github.com/kimdonghwi94/web-analyzer-mcp.git
cd web-analyzer-mcp

# Run directly with uv (auto-installs dependencies)
uv run mcp-webanalyzer

npx -y @smithery/cli install @kimdonghwi94/web-analyzer-mcp --client claude

{
 "mcpServers": {
 "web-analyzer": {
 "command": "uv",
 "args": [
 "--directory",
 "/path/to/web-analyzer-mcp",
 "run", 
 "mcp-webanalyzer"
 ],
 "env": {
 "OPENAI_API_KEY": "your_openai_api_key_here",
 "OPENAI_MODEL": "gpt-4"
 }
 }
 }
}

claude mcp add web-analyzer -e OPENAI_API_KEY=your_api_key_here -e OPENAI_MODEL=gpt-4 -- uv --directory /path/to/web-analyzer-mcp run mcp-webanalyzer

{
 "mcpServers": {
 "web-analyzer": {
 "command": "uv",
 "args": [
 "--directory",
 "/path/to/web-analyzer-mcp",
 "run", 
 "mcp-webanalyzer"
 ],
 "env": {
 "OPENAI_API_KEY": "your_openai_api_key_here",
 "OPENAI_MODEL": "gpt-4"
 }
 }
 }
}

{
 "mcpServers": {
 "web-analyzer": {
 "command": "uv",
 "args": [
 "--directory",
 "/path/to/web-analyzer-mcp",
 "run", 
 "mcp-webanalyzer"
 ],
 "env": {
 "OPENAI_API_KEY": "your_openai_api_key_here",
 "OPENAI_MODEL": "gpt-4"
 }
 }
 }
}

web-analyzer-mcp/
├── web_analyzer_mcp/ # Main Python package
│ ├── __init__.py # Package initialization
│ ├── server.py # FastMCP server with tools
│ ├── web_extractor.py # Web content extraction engine
│ └── rag_processor.py # RAG-based Q&A processor
├── scripts/ # Build and utility scripts
│ └── build.js # Node.js build script
├── README.md # English documentation
├── README.ko.md # Korean documentation
├── package.json # npm configuration and scripts
├── pyproject.toml # Python package configuration
├── .env.example # Environment variables template
└── dist-info.json # Build information (generated)

# Clone repository
git clone https://github.com/kimdonghwi94/web-analyzer-mcp.git
cd web-analyzer-mcp

# Development commands
uv run mcp-webanalyzer # Start development server
uv run python -m pytest # Run tests
uv run ruff check . # Lint code
uv run ruff format . # Format code
uv sync # Sync dependencies

# Install development dependencies
uv add --dev pytest ruff mypy

# Create production build
npm run build

# Setup Python environment (if not using uv)
pip install -e .[dev]

# Development commands
python -m web_analyzer_mcp.server # Start server
python -m pytest tests/ # Run tests
python -m ruff check . # Lint code
python -m ruff format . # Format code
python -m mypy web_analyzer_mcp/ # Type checking