Excalibur.Data.DynamoDb 3.0.0-alpha.208

Excalibur.Data.DynamoDb

AWS DynamoDB data provider implementation for Excalibur cloud-native data access.

Overview

This package provides a cloud-native data access implementation for AWS DynamoDB, implementing ICloudNativePersistenceProvider from Excalibur.Data.Abstractions.

Features

• Document Operations: Full CRUD support for DynamoDB documents
• Batch Operations: Efficient transactional batch creates and replaces using TransactWriteItems
• Query Support: Partition key and sort key queries with pagination
• Change Feed: DynamoDB Streams integration for real-time change data capture
• Health Checks: Built-in health check for monitoring connectivity
• Resilience: Configurable retry policies for transient failures

Installation

dotnet add package Excalibur.Data.DynamoDb

Configuration

Using Options Action

services.AddDynamoDb(options =>
{
 options.Connection.Region = "us-east-1";
 options.DefaultTableName = "MyTable";
 options.DefaultPartitionKeyAttribute = "pk";
 options.DefaultSortKeyAttribute = "sk";
});

Using Configuration Section

services.AddDynamoDb(Configuration.GetSection("DynamoDb"));

appsettings.json

{
 "DynamoDb": {
 "DefaultTableName": "MyTable",
 "DefaultPartitionKeyAttribute": "pk",
 "DefaultSortKeyAttribute": "sk",
 "UseConsistentReads": false,
 "Connection": {
 "Region": "us-east-1",
 "TimeoutInSeconds": 30,
 "MaxRetryAttempts": 3
 }
 }
}

Using Existing Client

If you already have an IAmazonDynamoDB client configured:

services.AddSingleton<IAmazonDynamoDB>(sp => new AmazonDynamoDBClient());
services.AddDynamoDbWithClient(options =>
{
 options.DefaultTableName = "MyTable";
});

Configuration Options

Root Settings (DynamoDbOptions)

Connection Settings (DynamoDbConnectionOptions)

Access via options.Connection.*:

DynamoDB Streams Settings

Stream View Types:

• KEYS_ONLY - Only the key attributes
• NEW_IMAGE - The entire item after modification
• OLD_IMAGE - The entire item before modification
• NEW_AND_OLD_IMAGES - Both the new and old item images

Usage

Basic Operations

public class MyService
{
 private readonly ICloudNativePersistenceProvider _provider;

 public MyService(ICloudNativePersistenceProvider provider)
 {
 _provider = provider;
 }

 public async Task<Order?> GetOrderAsync(string customerId, string orderId)
 {
 return await _provider.GetByIdAsync<Order>(
 orderId,
 new PartitionKey(customerId));
 }

 public async Task CreateOrderAsync(Order order)
 {
 await _provider.CreateAsync(
 order,
 new PartitionKey(order.CustomerId));
 }

 public async Task<IReadOnlyList<Order>> GetCustomerOrdersAsync(string customerId)
 {
 return await _provider.QueryAsync<Order>(new PartitionKey(customerId));
 }
}

Batch Operations

// Batch create
var items = new[] { order1, order2, order3 };
await _provider.BatchCreateAsync(
 items,
 item => new PartitionKey(item.CustomerId));

// Batch replace with optimistic concurrency
await _provider.BatchReplaceAsync(
 items,
 item => new PartitionKey(item.CustomerId),
 item => item.ETag);

Change Feed (DynamoDB Streams)

var subscription = await _provider.CreateChangeFeedSubscriptionAsync<Order>(
 "Orders",
 new ChangeFeedOptions
 {
 StartPosition = ChangeFeedStartPosition.Beginning,
 MaxBatchSize = 100
 });

await foreach (var change in subscription.ReadChangesAsync(cancellationToken))
{
 Console.WriteLine($"{change.EventType}: {change.DocumentId}");

 if (change.Document != null)
 {
 // Process the changed document
 }
}

Note: DynamoDB Streams must be enabled on the table for change feed to work.

Health Check

services.AddHealthChecks()
 .AddCheck<DynamoDbHealthCheck>("dynamodb");

Local Development

For local development with DynamoDB Local:

services.AddDynamoDb(options =>
{
 options.Connection.ServiceUrl = "http://localhost:8000";
 options.DefaultTableName = "LocalTable";
});

Key Patterns

This provider follows the single-table design pattern commonly used with DynamoDB:

• Partition Key (pk): Groups related items together for efficient access
• Sort Key (sk): Provides ordering within a partition and enables range queries

Documents are serialized to/from DynamoDB using JSON, with partition and sort keys automatically managed.

Transactional Support

Batch operations use DynamoDB's TransactWriteItems for atomic, all-or-nothing operations:

// All items are created atomically
await _provider.BatchCreateAsync(items, item => new PartitionKey(item.Id));

Error Handling

The provider maps DynamoDB exceptions to CloudOperationResult<T>:

var result = await _provider.UpdateAsync(order, partitionKey, etag);

if (!result.Success)
{
 switch (result.StatusCode)
 {
 case 400:
 Console.WriteLine("Validation error or bad request");
 break;
 case 404:
 Console.WriteLine("Item not found");
 break;
 case 409:
 Console.WriteLine("Conditional check failed (item already exists)");
 break;
 case 412:
 Console.WriteLine("Precondition failed (ETag/version mismatch)");
 break;
 case 429:
 Console.WriteLine("Provisioned throughput exceeded - retry with backoff");
 break;
 }
}

DynamoDB-Specific Exceptions

Authentication

DynamoDB supports multiple authentication methods:

IAM Role (Recommended for AWS)

// Uses default credentials chain (EC2 instance profile, ECS task role, etc.)
services.AddDynamoDb(options =>
{
 options.Connection.Region = "us-east-1";
 options.DefaultTableName = "MyTable";
});

Explicit Credentials

services.AddDynamoDb(options =>
{
 options.Connection.Region = "us-east-1";
 options.Connection.AccessKey = "YOUR_ACCESS_KEY";
 options.Connection.SecretKey = "YOUR_SECRET_KEY";
 options.DefaultTableName = "MyTable";
});

Environment Variables

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_REGION="us-east-1"

Best Practices

1. Use Single-Table Design - Store related entities in one table using partition/sort key patterns
2. Design for Access Patterns - Model your data based on query requirements, not relational concepts
3. Enable Consistent Reads Only When Needed - Eventually consistent reads use half the RCUs
4. Use Transactional Batches for atomic operations across items
5. Monitor Consumed Capacity - Track ConsumedCapacity in responses
6. Enable Streams for Event-Driven architectures and change data capture
7. Use Exponential Backoff for ProvisionedThroughputExceededException
8. Consider On-Demand Mode for unpredictable workloads

License

See LICENSE files in the repository root.