ING on Building a Cloud Native Bank

In today’s world, customers expect a superior experience. This means technology, now even more than before, has become an essential link to provide those always on seamless digital services which enable our customers to stay safe and secure.

Since ING, a global bank with over 37 million customers, has a history of adapting to change, we’re always aiming to be a step ahead. To stay ahead in a banking tech environment, you need to be very opinionated on how IT is applied.

But this is not always the case in the broader tech ecosystem we depend on, which has so many other stakeholders to satisfy and, therefore, too often security (let alone compliance) is an afterthought.

So, we expect our employees to challenge this tech ecosystem by showing it is possible to have better security and easier compliance. We didn’t become who we are by being a follower.

Our customers (and by extension the politicians they elect and the regulators they have instituted) rely on ING to deliver on the promises we make; trust is our license to operate.

And we try hard to avoid any outages which could erode this trust. Of course, any outage could be a dent in our professional pride.

It’s safe to conclude that as ING’s tech employees, we have plenty of incentives to build a better tech ecosystem for ING.

Let me share in a bit more detail what “being a bank” means from a tech perspective.

For all the buzz around ING’s tech over the past years, sometimes our image is greater than our actual delivery. Yes, we dared to take some big leaps in the early days of DevOps and Agile, and we harvested the consequences of that, both positive and negative.

The inverse is also most certainly true: working in a bank has a certain image, and while there are certainly valid reasons for that, people don’t always realize that banking tech systems are still today some of the most complex tech systems in this world. And you would be right to challenge banks if this is a flaw or a virtue.

For good or bad, the fact is by operating those systems for decades, banks have collected a significant amount of institutional knowledge on how to securely operate complex systems at scale, which not even today’s hyperscalers have caught up with (as the larger banks in this world have at least a 25-year head start).

This means most banks have something to offer to individuals, open source communities and other partners. Hence, ING decided to become more active in the tech community:

• Encouraging employees to use their full potential (e.g., by speaking at conferences and writing tech blogs);
• Enabling departments to open source code either developed for ING internal use or to contribute back to open source projects we consume and
• Sponsoring conferences, organizing meetups and other events.

But please don’t misinterpret us opening up for advocating complexity. On the contrary, we definitively have the desire to simplify our tech ecosystem.

We learned our lessons the hard way and know for sure we want to reduce complexity, get away from tightly coupled systems, unmanageable vendor lock-ins, obsolete (sometimes even self-maintained) components and so on — and rather today than tomorrow. But reality always kicks in, and IT transformations do take their time.

Nevertheless, like any other tech department in any other company in this world, in the end, we’re still learning and improving day by day.

Part of those improvements is rebuilding our legacy systems into cloud native systems. That has been a journey for ING we started around 2015 by thinking through the concepts of our then next-generation infrastructure offerings within the enterprise architecture department. How could we enable faster and easier adaption of new technologies in ING?

One thing was obvious: we would need to tear down the walls of our vaults, open up our systems and build digital platforms.

To be quite honest, we were looking into concepts like separating Runtime Hosting from Data Services. (Based on the 12 Factor paradigm, as well as the work of Kolb & Wirtz: Towards Application Portability in Platform as a Service, University of Bamberg. It quickly became known internally as “The Bamberg Model.”) And we contemplated an “API-PaaS” delivery (something like Cloud Foundry) for our developers.

Then we experienced the Agile revolution within ING’s infrastructure departments, and our ideas of protecting Developers against themselves by limiting degrees of freedom and prescribing infrastructure patterns went down the drain. The result of these revised insights was a “serverless” Kubernetes-Namespace-as-a-Service (“NaaS”) delivery model in which Developers are fully responsible for everything they do within their namespaces. This NaaS is a globally useable building block providing a modular and scalable foundation to host ING’s immutable workloads. And it was born out of a collaboration between ING’s Polish, German and Dutch engineers.

As a result, some DevOps teams building and managing ING’s applications flourished, while others struggled with the cognitive load of these freedoms and responsibilities. Sadly, this learning experience did cause us some outages which might have been avoidable, in hindsight.

Debates with teams who want cluster-level privileges to run their applications (and are de facto asking for dedicated Kubernetes clusters) and teams who find it too hard to consume and operate namespaces and would prefer to have an API-PaaS style delivery or a Functions as a Service are still common, even with this NaaS operating model.

The other reality we had — and have — to deal with is a scarcity of engineering resources. We couldn’t realistically develop and maintain both an “API-PaaS” and a “NaaS” model simultaneously (let alone the other models mentioned), especially since we initially did not have a large volume to make a business case with.

In the end, everybody involved was a bit right and a bit wrong. The most important lesson here is that a one-size-fits-all operating model will only work if the organization around it is aligned with it and supports its developers to work in that operating model.

Fast forward to today:

ING is looking to assist its developers with a private cloud offering standardized services like the already mentioned Kubernetes NaaS. That NaaS is provided by the second generation of ING’s Container Hosting Platform (ICHPv2). ING builds 36 ICHPv2 components to create that NaaS and make it fully automated.

We call the architecture behind ICHPv2 “Zero-Privilege,” and it will be presented publicly at KubeCon + CloudNativeCon EU (April 19-21 2023) in ING’s corporate hometown of Amsterdam. During that same conference, ING will open source the first three NaaS components under the Neoria (“Dockyard”) brand at the ING booth:

• Neoria Skavos: An Operator Framework for Python based upon observer design pattern
• Neoria Orchestration Package: Orchestration over Clusters, interfacing with Private Cloud Orchestrator
• Neoria Quota AutoScaler: Sets namespace quota according to deployment limits

These components have enabled ING to significantly reduce our CPU usage and hence our CO2 footprint. And since ING is putting sustainability at the heart of what we do, we make this code available to the rest of the world so even more CPU cycles can be saved and corresponding CO2 exhaust avoided.

But we’re only getting started.

Accompanying the “Zero Privilege Architecture” talk, there will be a second ING talk, “Kubernetes: Resistance is Futile,” from a presenter actually using this Private Cloud ecosystem.

During various pre-conferences, ING speakers will also share their expertise with the audiences:

• Observability Day (“Banking Observability at Scale”)
• OpenShift Commons (“Workload deployment Quality”)
• Data-on-Kubernetes (“Local persistence for Data Services at scale in ING”)

At the ING booth, we have a multitude of interesting “Booth Talks” ranging from the workload configuration templating services which are offered on top of NaaS (“King’s Road”) and ING’s proprietary Service Mesh (“Touch Mesh”) to ING’s future hybrid cloud (“Public Cloud Foundation/Paved Roads”) and many more.

There will also be scheduled visits of all the ING speakers from KubeCon and its pre-conferences. In case you didn’t get to ask questions after the talk or missed the talk entirely and regret that, here’s your second chance!

Last but not least, the Chairman of ING’s Open Source Board will be at the ING booth sharing how ING is evolving from a consumer to a contributor in the ecosystem.

We hope this article and our presentations at KubeCon will give some insights into what it means to be in a banking tech environment and how to transform into a Cloud Native bank.

Obviously, there’s much more to share than we have space for in this article.

If you are in the opportunity to travel to Amsterdam, we hope to speak to you during KubeCon EU and hear your feedback. And even if you do not work for a bank, feel free to approach us and learn how to improve tech(-security) ecosystems in general, wherever you’re employed.

The artwork in this presentation (“Opening Up” and visuals Cloud Native ecosystem Kube) is from my esteemed colleague, Theo Sommer.