AppPipe.Hosting 2.0.5

AppPipe.Hosting 🚀

👁 NuGet Version
👁 NuGet Downloads
👁 License

AppPipe is a lightweight, zero-container-dependency alternative to the .NET Aspire dashboard and gateway runner.

While .NET Aspire is highly optimized for cloud-first deployments and relies heavily on local Docker/container runtimes, AppPipe is built specifically for traditional virtual machines, bare-metal servers, and on-premises environments across Windows (IIS / Windows Services), Linux (systemd / Nginx / Caddy), and macOS.

With AppPipe, you get a beautiful, unified developer dashboard, OpenTelemetry (OTLP) collection, and service discovery routing without the container dependencies or cloud-only overhead.

🌟 Features

• 📊 OpenTelemetry Collector & Dashboard: Collects OTLP traces, logs, and metrics. Displays them in a gorgeous HTML5/Razor Pages dashboard (complete with Light/Dark modes, trace waterfall flamegraphs, structured console logs, and metric charts).
• 💾 Extensible Database Persistence: Transitive out-of-the-box telemetry storage via a local SQLite database, with full support for enterprise backends like PostgreSQL, ClickHouse, SQL Server, MySQL, or Elasticsearch, as well as a lightweight circular in-memory buffer.
• 🔒 Dashboard Security: Opt-in basic authentication protection for all dashboard and diagnostics routes.
• 📈 Gateway Diagnostics Panel: A dedicated diagnostics page showing real-time telemetry ingestion rates, active proxy connections, database sizes, and host system information.
• 🔄 Unified Gateway & Routing: Powered by YARP (Yet Another Reverse Proxy), AppPipe hosts a central routing gateway that automatically maps and proxies requests to your backend microservices.
• 🔌 Dynamic Port Allocation: Automatically assigns free ports to your applications during local runs or deployment pipelines, preventing port conflict issues.
• 🏢 Multi-Platform Deployment Engines: Standard built-in modules powered by ModularPipelines to automate publishing, creating Windows AppPools, registering IIS sub-applications, setting up Linux systemd units, and generating Nginx/Caddy configurations.
• ⚡ Auto-Refresh Controls: Fully control dashboard background polling using the integrated Auto-Refresh toggle (with configurable timing) to prevent unnecessary CPU/network overhead.

📖 Detailed Documentation

For an in-depth dive into how AppPipe works under the hood and how to configure it for production deployment, refer to the .

It covers:

• ⚙️ Fluent Topology Options: Detailed documentation for .WithEndpoint(), .WithAppPool(), .WithServiceAccount(), .WithReference(), and other builder methods.
• 🏢 On-Premises IIS & Service Deployment: Technical details regarding AppPool permissions, sub-application path matching, self-healing file locks, and the IIS token overwrite startup filter.
• 🐧 Linux systemd & Reverse Proxy Layouts: Guidance and auto-generated configurations for systemd units, Nginx location blocks, and Caddy directives.
• 🚀 DevOps CI/CD Integration: Guidelines on using --prepublished-dir to deploy pre-compiled DLLs directly via GitHub Actions or Azure DevOps pipelines without requiring the .NET SDK on the target server.
• 🛠️ CLI Troubleshooting & Diagnostic Commands: PowerShell and CMD commands for testing IIS sites, AppPool recycles, port conflict resolution, and enabling standard output logging.

⚙️ Architecture

graph LR
 %% Styles
 classDef client fill:#f97316,stroke:#ea580c,stroke-width:2px,color:#fff;
 classDef gateway fill:#0284c7,stroke:#0369a1,stroke-width:2px,color:#fff;
 classDef service fill:#10b981,stroke:#047857,stroke-width:2px,color:#fff;
 classDef telemetry fill:#8b5cf6,stroke:#6d28d9,stroke-width:2px,color:#fff;
 classDef store fill:#6366f1,stroke:#4f46e5,stroke-width:2px,color:#fff;
 classDef dashboard fill:#ec4899,stroke:#db2777,stroke-width:2px,color:#fff;

 Client[Browser / API Client]:::client
 
 subgraph AppPipeHost ["AppPipe Host Application"]
 Gateway[YARP Gateway Proxy]:::gateway
 TelemetryPort[Telemetry gRPC Port]:::telemetry
 Store[(In-Memory Store)]:::store
 Dashboard[Dashboard UI]:::dashboard
 
 TelemetryPort -->|Write| Store
 Store -->|Read| Dashboard
 end

 subgraph Microservices ["User Microservices Space"]
 ServiceA[Microservice A]:::service
 ServiceB[Microservice B]:::service
 end

 %% Request Routing Flow
 Client -->|1. HTTP Request| Gateway
 Gateway -->|2. Route proxy| ServiceA
 Gateway -->|2. Route proxy| ServiceB

 %% Telemetry Flow
 ServiceA -.->|3. OTLP Traces/Logs/Metrics| TelemetryPort
 ServiceB -.->|3. OTLP Traces/Logs/Metrics| TelemetryPort

Install the NuGet Packages

To add the library to an existing project:

dotnet add package AppPipe.Hosting

📦 Project Scaffolding Templates

AppPipe provides a custom .NET template pack that scaffolds a fully working multi-project solution structure out of the box (including the AppHost orchestrator, an ApiService backend, and a Web frontend):

1. Install the Template Pack

dotnet new install AppPipe.Hosting.Templates

2. Scaffold a New System Solution

Create a new directory for your microservices solution and run:

dotnet new app-pipe -n MySystem

This generates:

• MySystem.sln: The Visual Studio solution file.
• MySystem.AppHost: The AppPipe orchestrator and gateway dashboard.
• MySystem.ApiService: A backend REST API configured with OpenTelemetry.
• MySystem.Web: A frontend web application that calls the backend using dynamic service discovery.

⚙️ Template Configuration Choices

When scaffolding with dotnet new app-pipe, you can customize your architecture, frontend, database, auth, and caching options:

For example, to scaffold a full production CQRS architecture with a Blazor frontend, SQLite database, secure JWT authorization, and Redis caching:

dotnet new app-pipe -n MySystem --architecture clean-cqrs --database sqlite --auth jwt --caching redis

🔌 Integrating AppPipe into Existing Solutions

If you already have an existing .NET microservices solution and want to add AppPipe orchestration, dashboarding, and telemetry, follow these steps:

Step 1: Create the AppHost Orchestrator Project

Add a new empty .NET Console or Web application project named YourSolution.AppHost to your existing solution:

dotnet new web -n YourSolution.AppHost

Step 2: Add Package and Project References

1. Install the AppPipe.Hosting package to the new AppHost project:

   dotnet add YourSolution.AppHost/YourSolution.AppHost.csproj package AppPipe.Hosting
   

2. Reference your existing microservices from the AppHost project using standard Project References:

   dotnet add YourSolution.AppHost/YourSolution.AppHost.csproj reference YourExisting.Backend/YourExisting.Backend.csproj
   dotnet add YourSolution.AppHost/YourSolution.AppHost.csproj reference YourExisting.Frontend/YourExisting.Frontend.csproj
   

Step 3: Write the Orchestration Entry Point

Replace the contents of Program.cs in the YourSolution.AppHost project with:

using AppPipe.Hosting;

var builder = AppPipeHostingApp.CreateBuilder(args);

// Register your referenced projects:
// Note: AppPipe automatically generates compile-safe constants for your projects
// (e.g. AppPipeProjects.YourExisting_Backend) during compilation!
var backend = builder.AddProject(AppPipeProjects.YourExisting_Backend)
 .WithEndpoint(5001); // Assign an entry endpoint port

var frontend = builder.AddProject(AppPipeProjects.YourExisting_Frontend)
 .WithEndpoint(5002)
 .WithReference(backend); // Automatically injects discovery environment variables

var app = builder.Build();

// Run the local orchestrator and OTLP collector
var runner = new AppPipeDevHostRunner(app);
await runner.RunAsync();

Step 4: Configure OpenTelemetry in Your Existing Services

In each of your child microservices, add OpenTelemetry OTLP exporters. AppPipe automatically injects the OTLP telemetry ports and service discovery variables as environment values into your processes at run-time:

1. Install the OpenTelemetry packages in your child projects:

   dotnet add package OpenTelemetry.Extensions.Hosting
   dotnet add package OpenTelemetry.Instrumentation.AspNetCore
   dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
   

2. Configure it in their Program.cs:

   builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
    .AddAspNetCoreInstrumentation()
    .AddOtlpExporter()) // Automatically picks up AppPipe gRPC port
    .WithMetrics(metrics => metrics
    .AddAspNetCoreInstrumentation()
    .AddOtlpExporter()); // Automatically picks up AppPipe gRPC port
   

🚀 Quick Start

1. Define your App Topology

Configure your services and their relationships in your entry point:

using AppPipe.Hosting;

var builder = AppPipeHostingApp.CreateBuilder(args);

// Define a backend worker microservice using compile-safe generated constant
var backend = builder.AddProject(AppPipeProjects.BackendWorker);

// Or register directly using raw string name:
// var backend = builder.AddProject("BackendWorker");

// Define a frontend API that communicates with the backend
var frontend = builder.AddProject(AppPipeProjects.FrontendApi)
 .WithReference(backend); // Injects service discovery variables automatically


var app = builder.Build();

// Run the host using AppPipeDevHostRunner
var runner = new AppPipeDevHostRunner(app);
await runner.RunAsync();

2. Configure telemetry in your Microservices

In your microservices, register the standard OpenTelemetry exporter. They will automatically detect the telemetry endpoints exposed by the AppPipe Gateway.

builder.Services.AddOpenTelemetry()
 .WithTracing(tracing => tracing
 .AddAspNetCoreInstrumentation()
 .AddHttpClientInstrumentation()
 .AddOtlpExporter()) // Exports to AppPipe telemetry port
 .WithMetrics(metrics => metrics
 .AddAspNetCoreInstrumentation()
 .AddHttpClientInstrumentation()
 .AddOtlpExporter());

🖥️ Accessing the Dashboard

Once the AppHost is running, open your browser and navigate to:

http://localhost:7001/dashboard

The dashboard port 7001 is the default gateway port. If you have changed it in your appsettings.json, use the port you configured instead.

What you'll see

Starting with the template

If you used the AppPipe template:

# In Visual Studio — set the AppHost project as the startup project and press F5
# Or via CLI:
cd MySystem.AppHost
dotnet run

Then open http://localhost:7001/dashboard in your browser.

You can customize the dashboard, security, and persistence behavior in your appsettings.json or environment variables:

{
 "Dashboard": {
 "UseWebSockets": false,
 "BasicAuth": {
 "Enabled": true,
 "Username": "admin",
 "Password": "MySecretPassword"
 }
 },
 "Telemetry": {
 "PersistenceEnabled": true,
 "DatabasePath": "telemetry.db",
 "MaxDbRecords": 2000
 }
}

🔄 Configuring the Gateway via YARP

AppPipe's gateway binds natively to standard .NET configuration under the "ReverseProxy" section. You can define custom routes, clusters, HTTP request properties, rate-limiting, and transforms inside your appsettings.json, or programmatically configure them using .ConfigureGateway() in Program.cs.

For a complete guide, step-by-step code samples, and auto-generated routing details, see the .

🌐 Server-Side URL Resolution

To support various environments and multi-server topologies, the dashboard dynamically computes absolute URLs for registered microservices on the server side:

• Development Heuristics: Automatically treats environments like Development or LocalDev as local debugging sessions, outputting direct Kestrel ports (e.g., http://localhost:5001) to bypass gateways.
• IIS Site Sharing: Detects if the dashboard and child apps share the same IIS Web Site name (e.g. "Default Web Site"), resolving sub-application paths automatically: scheme://Request.Host{AppPath}.
• Standalone Ports: Falls back to dedicated host ports for standalone websites, Windows Services, and Linux daemons: scheme://Request.Host.Host:{AssignedPort}.
• Dynamic Scale: Resolves links using visiting HTTP headers, allowing seamless access via private IPs, load-balancer DNS, or domain aliases.
• Configuration Overrides: Manually configure explicit endpoints (AppPipe:Endpoints:{Resource}) or force gateway URLs using AppPipe:UseGatewayUrls.

For detailed information, see the .

💾 Extensible Telemetry Storage & Custom Databases

By default, AppPipe persists logs, traces, and metrics in a local SQLite database (). This provides zero-configuration persistence out of the box.

However, SQLite is strictly a default option. AppPipe is fully database-agnostic. You can easily configure it to run entirely in-memory (using a ring buffer) or persist to enterprise databases like PostgreSQL, ClickHouse, SQL Server, MySQL, or Elasticsearch by implementing the interface and registering it:

builder.ConfigureGateway(gatewayBuilder =>
{
 // Override the default SQLite persistence with your custom store
 gatewayBuilder.Services.AddSingleton<ITelemetryStore, ClickHouseTelemetryStore>();
});

To run purely in-memory, set Telemetry:PersistenceEnabled to false in your appsettings.json.

For details and step-by-step code examples, see the .

🏢 On-Premises Deployment

AppPipe includes a built-in deployment module utilizing ModularPipelines to automate publishing and deployments directly to IIS, Windows Services, or Linux systemd.

1. Customizing Deployment Properties (Fluent Builder)

You can configure environment-specific settings (such as custom IIS AppPools, sites, display names, startup accounts, and hosting models) directly in your orchestrator topology configuration:

var backend = builder.AddProject("BackendWorker")
 // IIS & Linux Reverse Proxy Settings
 .WithAppPool("CustomBackendPool")
 .WithIISSite("Default Web Site")
 .WithAppPath("/backend") // Custom path (IIS virtual path, Nginx location, or Caddy handle_path)
 .WithHostingModel("OutOfProcess") // "InProcess" or "OutOfProcess"
 
 // Windows Service / systemd Settings
 .WithServiceDisplayName("AppPipe Backend Worker Service")
 .WithServiceDescription("AppPipe backend processing service runs tasks.")
 .WithServiceStartType("auto") // "auto", "demand", or "disabled"
 .WithServiceAccount(@"DOMAIN\user")
 .WithServicePassword("secret_password");

2. Customizing the Dashboard (Host Project)

The dashboard itself is represented as builder.HostProject (an instance of AppPipeHostingProjectResource) and can be named and configured just like any other microservice:

// Set a custom dashboard application name (used for SCM Service name)
builder.HostProject = new AppPipeHostingProjectResource("AppPipeDashboard", "");

// Configure the dashboard options fluently
builder.HostProject.WithEndpoint(7001)
 .WithIISSite("Default Web Site")
 .WithAppPath("/") // Deployed directly at the root '/' of the site/proxy
 .WithAppPool("AppPipeDashboardPool")
 .WithServiceDisplayName("AppPipe Dashboard Orchestrator")
 .WithServiceDescription("AppPipe gateway and diagnostic telemetry UI.");

3. Running Local Deployments

To deploy the gateway and microservices directly from your development machine:

# Deploy to IIS under a custom sub-path
dotnet run --project YourDevHost.csproj -- --deploy iis /app-pipe-host-test

# Deploy as Windows Services
dotnet run --project YourDevHost.csproj -- --deploy windows-service

🚀 DevOps CI/CD Pipelines (Deploying Pre-Compiled DLLs)

In a typical CI/CD pipeline, the build agent compiles the code (creating DLL artifacts), and the release agent downloads the pre-compiled files onto the target environment where no source code or .csproj files exist.

AppPipe supports this via the --prepublished-dir flag and configuration binding.

1. CI Stage (Build)

Compile and publish your projects into a target directory (e.g. ./publish):

# Publish DevHost (Orchestrator) and child projects
dotnet publish samples/AppPipe.DevHost/AppPipe.DevHost.csproj -c Release -o ./publish/AppPipe.DevHost
dotnet publish samples/BackendWorker/BackendWorker.csproj -c Release -o ./publish/BackendWorker
dotnet publish samples/FrontendApi/FrontendApi.csproj -c Release -o ./publish/FrontendApi

Upload the ./publish directory as a build artifact.

2. CD Stage (Deploy)

Download the published artifact to the target server and execute the orchestrator pointing to the pre-compiled folder. This completely bypasses source code compilation and .csproj file searches:

dotnet C:\inetpub\apps\AppPipe\AppPipe.DevHost.dll --deploy iis --prepublished-dir C:\inetpub\apps\AppPipe

3. Handling Environment-Specific Configs & Secrets

To avoid hardcoding values (like AppPool names, Service Accounts, or database passwords) in Program.cs, bind your orchestrator to .NET configuration (IConfiguration):

// Program.cs
var config = new ConfigurationBuilder()
 .AddJsonFile("appsettings.json", optional: true)
 .AddEnvironmentVariables(prefix: "APPIPE__")
 .AddCommandLine(args)
 .Build();

var appPool = config["BackendWorker:AppPoolName"] ?? "DefaultPool";
var password = config["BackendWorker:ServicePassword"]; // Read securely

builder.AddProject("BackendWorker")
 .WithAppPool(appPool)
 .WithServiceAccount(@"DOMAIN\ServiceAccount")
 .WithServicePassword(password);

Injecting Values securely in your CD pipeline:

• As Environment Variables: Map pipeline variables or secrets as environment variables prefixed with APPIPE__:
  • APPIPE__BackendWorker__AppPoolName $\rightarrow$ ProductionPool
  • APPIPE__BackendWorker__ServicePassword $\rightarrow$ $(SecretServicePasswordValue)
• As Command-line Arguments:

  dotnet AppPipe.DevHost.dll --deploy iis --prepublished-dir C:\inetpub\apps\AppPipe --BackendWorker:AppPoolName "ProductionPool" --BackendWorker:ServicePassword "$(SecretServicePasswordValue)"
  

🛠️ Developer Guide: Working on the Repository

This section guides developers on how to work inside the AppPipe.Hosting repository, compile the projects, run sample environments, and package/test NuGet templates.

1. Prerequisites

• .NET 10.0 SDK or higher
• PowerShell 7 (recommended for package automation)

2. Repository Layout

• AppPipe.Hosting/: Core library (Razor Pages pages, YARP configuration, OTLP listener, SQLite store, process manager).
• templates/AppPipeSystemTemplate/: Scaffolding source code packaged as .NET templates.
• samples/: Test projects (AppPipe.DevHost, BackendWorker, FrontendApi) used to test the gateway runner and telemetry collection.
• tests/: Unit and integration test suites.

3. Local Run & Debugging

To launch the developer sample environment locally (runs on Windows, Linux, and macOS):

# Navigate to the DevHost project
cd samples/AppPipe.DevHost

# Start the DevHost orchestrator
dotnet run

This command compiles and launches the gateway proxy, boots the microservices on dynamic local ports, registers them, and configures OTLP telemetry back to the dashboard at http://localhost:7001/dashboard.

4. Generating NuGet Packages

You can pack the AppPipe gateway library and its templates pack locally using the standard .NET CLI:

# Build the solution in Release mode
dotnet build -c Release

# Pack the core hosting library (generates AppPipe.Hosting.<version>.nupkg in bin/Release)
dotnet pack AppPipe.Hosting/AppPipe.Hosting.csproj -c Release

# Pack the template pack (generates AppPipe.Hosting.Templates.<version>.nupkg)
dotnet pack AppPipe.Hosting.Templates.csproj -c Release

5. Installing and Testing Local Templates

To verify template edits locally before uploading to NuGet:

# Register the templates pack from the local folder
dotnet new install templates/AppPipeSystemTemplate

# Check that app-pipe template is listed
dotnet new list app-pipe

# Test scaffolding in a clean folder
mkdir MyTestSystem
cd MyTestSystem
dotnet new app-pipe -n MyTestSystem

📄 License

This project is licensed under the MIT License - see the file for details.