Measuring the Impact of Your Platform Engineering Strategy

A key tenet of platform engineering is to treat the platform like a product and your colleagues who use it like your customers.

Anyone who has managed a product will tell you that you need engagement metrics to inform your product strategy. The same should be said about platform engineering. Just like with a customer-facing product, your platform needs to deliver value to win users over their trusted alternatives.

In our work with our customers, we’ve found that many platform teams need a mechanism to measure user engagement in a practical way that will inform the product strategy supporting the platform.

Here is what we’ve told these customers.

Expect Friction Around Platform Adoption

It’s important to consider that your platform may face resistance and several alternative suggestions.

In many cases, the teams you are designing your platform to support have spent years executing their day-to-day work with alternative tools and platforms. Be it DevOps teams working in Infrastructure as Code (IaC), development teams that rely on CI/CD platforms or IT teams using the native AWS console, you need to consider whether your platform offers an advantage to the status quo.

Monitor User Engagement with Your Platform

We suggest monitoring day-to-day activity with the resources delivered through the platform.

As the orchestration layer for application infrastructure, our Torque platform integrates with internal developer platforms (IDPs) to automate the deployment of cloud resources and environments supporting the development lifecycle.

Every time a developer deploys an application environment through the IDP, our platform orchestrates and launches the cloud resources supporting that environment. Based on that action, our users can see which application resources are deployed, by whom (by user and team) and for what purpose (by tags applied to cloud resources upon launch).

So if, for example, activity among one of the teams you support declines or lags behind that of their colleagues, you can investigate as soon as the trend emerges.

Visibility into the resource configurations in Git allows you to verify that the resources were orchestrated correctly for the workload. Meanwhile, insight into the users who can access those resources tells you who to talk to for feedback that would improve the experience.

This is the starting point to identify any friction that could hold back developer experience and to gather feedback from the users who are most affected.

Solicit Recommendations to Optimize Your Platform

With this data, you can start to understand how the teams who rely on your platform leverage application resources.

Here are some questions to ask that can help improve adoption:

• What kinds of integrations might improve the user experience? In many cases, IDPs enhance IaC, CI/CD, IT service management (ITSM) or other operational tools that your colleagues prefer. If you identify a lapse in experience, extending support for existing tools and workflows could be a place to start.
• Is infrastructure uptime and resiliency disrupting the platform experience? Similar to a customer-facing product, the most important ability for your platform is availability. Gather feedback on failures or other performance issues that may have diminished trust in your platform, and explore the resources the platform delivers to identify the cause.
• Do users have access to documentation on application resources? Navigating a new platform could be disorienting for teams that are conditioned to work with traditional tools and approaches. Make sure that all application resources include documentation so that your teams can validate that it’s the right infrastructure for their workloads.

And remember — just like a product, a platform is never finished. Visibility into engagement is critical to maintaining and improving your platform experience over time.