In the Great Microservices Debate, Value Eats Size for Lunch

In May, an old hot topic in software design long thought to be settled was stirred up again, sparked by an article from the Amazon Prime Video architecture team about moving from serverless microservices to a monolith. This sparked some spirited takes and also a clamor for using the right architectural pattern for the right job.

Two interesting aspects can be observed in the melee of ensuing conversations. First, the original article was more about the scaling challenges with serverless “lambdas” rather than purely about microservices. Additionally, it covered state changes within Step Functions leading to higher costs, data transfer between lambdas and S3 storage, and so on.

It bears reminding that there are other and possibly better ways of implementing microservices other than just the use of serverless. The choice of serverless lambdas is not synonymous with the choice of microservices. Choosing serverless as a deployment vehicle should be contingent upon factors such as expected user load and call frequency patterns, among other things.

The second and more interesting aspect was about the size of the services (micro!) and this was the topic of most debates that emerged. How micro is micro? Is it a binary choice of micro versus monolith? Or is there a spectrum of choices of granularity? How should the size or granularity factor into the architecture?

Value-Based Services: Decoupling to Provide Value Independently

A key criterion for a service to be standing alone as a separate code base and a separately deployable entity is that it should provide some value to the users — ideally the end users of the application. A useful heuristic to determine whether or not a service satisfies this criterion is to think about whether most enhancements to the service would result in benefits perceivable by the user. If in a vast majority of updates the service can only provide such user benefit by having to also get other services to release enhancements, then the service has failed the criterion.

Services Providing Shared Internal Value: Coupling Non-Divergent Dependent Paths

What about services that offer capabilities internally to other services and not directly to the end user? For instance, there might be a service that offers a certain specialty queuing that is required for the application. In such cases, the question becomes whether the capabilities provided by the service have just one internal client or several internal clients.

If most of the time a service ends up calling exactly just one other service apart from very few exceptional cases where-in the call path may diverge, then there is little benefit in separating that service and its most predominant dependency. Another useful heuristic:  if a circuit breaks and a service is unable to reach one of its dependency services, can the calling service provide anything at all to its users or nothing?

Avoiding Talkative Services with Heavy Payloads

Providing value is also about the cost efficiency of designing as multiple services versus combining as a single service. One such aspect that was highlighted in the Prime Video case was chatty network calls. This could be a double whammy because it not only results in additional latency before a response goes back to the user, but it might also increase your bandwidth costs.

This would be more problematic if you have large or several payloads moving around between services across network boundaries. To mitigate, one could consider the use of a storage service, so one doesn’t need to move the payload around, rather only an identifier of the payload and only services that need it to consume it.

However even if an ID is passed around, if several services along the call path need to inspect or operate on the payload, those would need to pull the payload down from the storage service which would completely nullify and possibly worsen the situation.

How and where payloads are handled should be an important part of designing service boundaries and thereby influencing the number of services we have in the system.

Testability and Deployability

Finally, one more consideration would be the cost of rapidly testing and deploying services. Consider a scenario wherein a majority of the time multiple services need to be simultaneously enhanced in order to provide a feature enhancement to the user.

Feature testing would involve testing all of those services together. This could potentially result in bottlenecks for releases or necessitate the requirement for complex release control and testing mechanisms such as feature flags or blue-greening sets of services, among other things. This tendency is a sure-shot sign of the disadvantageous proliferation of too many discrete parts.

Too many teams fall into the trap of building “service enhancements” in every release but those enhancements not doing much for the end user because a number of other pieces from other services need to come together. Such highly coupled architecture complicates both dependency management and versioning, with delays in delivering “end user value”.

Value-Based Services, Not ‘Micro’  Services!

Architecture should be able to deliver value to the end users a majority of the time by the release of individual services independently. Considerations of coupling, dependencies, ease of testing and frequency of deployment matter more, while the size of the service itself has limited usefulness other than for applying reasonable limits on becoming too gigantic or too nano-sized.

There may be other esoteric reasons for splitting or creating multiple services such as the way our teams are organized (Conway’s law, anyone?) or providing flexibility with languages and frameworks but these are rarely real needs for providing value in enterprise software development.

One could very well have a performant cost-efficient architecture that delivers “value” with a diverse mix of services of various sizes — some big, some micro, and others somewhere in between. Think of it as a “value-based services architecture” rather than a “microservices-based architecture” that enables services to deliver value quickly and independently. Because value always eats size for lunch!