Pricing

Rating

Developer

Actor stats

Categories

Share

Incremental Data Sync Engine

Sync data changes from websites incrementally. Detect added, modified, and deleted items between runs. Perfect for monitoring product catalogs, listings, and any structured data.

🎯 Features

• Change Detection: Track added, modified, and deleted items
• Multiple Data Types: JSON API, HTML tables, HTML lists, XML feeds
• Field Tracking: Monitor specific fields for changes
• State Persistence: Automatically save state between runs
• Flexible Output: Changes only, incremental, or full sync

📥 Input

{
"sourceUrls":[{"url":"https://api.example.com/products"}],
"dataType":"json",
"identifierField":"id",
"trackFields":["price","status","quantity"],
"syncMode":"changes-only",
"jsonPath":"data.products",
"maxItems":10000
}

📤 Output (changes-only mode)

{
"id":"SKU-123",
"changeType":"modified",
"changedFields":[
{
"field":"price",
"oldValue":99.99,
"newValue":89.99
}
],
"oldItem":{"id":"SKU-123","price":99.99},
"newItem":{"id":"SKU-123","price":89.99},
"detectedAt":"2024-01-15T10:30:00Z"
}

🔄 Sync Modes

💡 Use Cases

1. Price Monitoring: Track product price changes
2. Stock Tracking: Monitor inventory changes
3. Content Updates: Detect new/updated articles
4. Competitor Analysis: Track listing changes
5. Data Warehousing: Sync only deltas to reduce storage

📊 Pricing

Suggested: $0.0001 per change detected

⚙️ How It Works

1. Fetch data from source URLs
2. Compare with previous run's data
3. Detect added/modified/deleted items
4. Output changes based on sync mode
5. Save current data for next run