Several leading financial services firms with well established public cloud programs have broadened the focus of their strategies from IaaS to PaaS services. This shift is driven by their desire to exploit new platforms (e.g. data science stacks) and improve developer productivity. The higher level abstraction involved with PaaS brings to the fore certain challenges that were solvable in relatively traditional ways with IaaS but require the adoption of cloud native approaches to open up the benefits of PaaS. In this blog, we will scratch the surface of those challenges and then delve into further detail in subsequent articles.
For IaaS, many of the industry’s stringent control requirements were solved by applying tight controls over the cloud network perimeter, building abstraction over the CSP control plane and embedding controls into the Operating System builds, augmented with the necessary cloud native controls needed to build a full service. Below, we describe why PaaS needs to be treated differently.
Managing workload authentication and authorization well is the number one line of defence when utilising PaaS services. Most financial services firms already have mature workload identity controls in place inside their four walls, but in many cases, the solutions do not translate well into a cloud environment with a fast rate of turnover. The traditional approaches are also often unsupported by the CSP. This is less of an issue for IaaS due to the deep level of ownership of the stack by the client, but it becomes important to address with PaaS where there are no integration hooks into traditional identity and auth systems.
Infrastructure as Code and Policy Enforcement
To support IaaS, many firms embarked on building or integrating a Cloud Management Platform acting as an abstraction layer over the Cloud Service Provider(s), into which policies and controls could be embedded and much of the CSP inner working could be hidden from end users. Given the heterogeneous nature and rapid feature velocity of PaaS, such an abstraction layer becomes extremely difficult and costly to maintain, reduces agility and degrades value. Instead, breaking out the cross-cutting aspects that require central management, automating those centrally and then encoding policy into the cloud allows application developers the freedom to use services as they were intended within the guardrails set by policy with minimal friction.
Many firms mandate the use of keys controlled by the firm (not the CSP), even for low sensitivity data. Integration with keys stored in the customer-managed key vault service is often a latent feature for CSPs when launching a new service and may limit the uptake of services by financial services firms unless other techniques, such as tokenization or application-level encryption, can be engineered into the application. These techniques themselves may not be easily achievable with certain services or for certain use cases. With IaaS, the encryption requirements could be engineered by the client into the operating system and middleware components, the configuration of which was the responsibility of the client.
Porous Network Boundary
The three main CSPs were born for the Internet, meaning that services are designed to enable public internet access (both in- and outbound). the ability to fully restrict access for these services to specific private networks comes late in the product lifecycle, if at all. Financial services firms are working closely with their CSP partners to prioritize and drive this “privatization” of services. In the meantime, complex network engineering is required, which in many cases constrains certain features and requires acceptance of greater security risk to achieve an acceptable level of compliance. Other controls need to be much stronger to compensate for this porous network boundary, which has for so long been the first line of defense against external threats.
In an IaaS-centric world, developers can still develop and unit test on their local machines, run their build and integration tests in an on-premises pipeline and then ship built artifacts into the cloud for later phases of testing and production deployment. Because a PaaS offering is, for the most part, available only at a single, remote CSP, there is no choice but to perform activities at each stage of the development pipeline in the cloud. Constantly shipping code and artifacts back and forth between on-premises and the cloud is inefficient, particularly if there are large executable and data artifacts, so the closer the development pipeline is to the services and data being used the more this opens up the agility benefits of the cloud to developers. Even if on-premise versions can be provided, often there are CSP customizations which in the DevOps “shift left” model should be exposed early in the development pipeline to avoid unnecessary delay and costly refactoring.
Observability and Monitoring
As “Polycloud” takes hold and systems become increasingly distributed and fragmented, having the ability to aggregate and correlate logs, metrics and events across several layers of disparate, evolving systems and to trace complex transactions throughout several PaaS service transformations is challenging. Even within a single CSP, most PaaS services generate their own log and metric data structures with poor out-of-the-box integration with the CSP’s monitoring services. Adoption of open standards for logging, metrics and tracing can help here, however many of these are still in the early phases of development and adoption by the CSPs is limited. As firms begin to look at systems with a higher bar for audit solving these challenges will become a barrier to adoption. Interestingly, many of the techniques learnt through years of managing market data and trading systems have analogous challenges to those faces with cloud native systems and should put banks who can tap into their strong heritage in capital markets in a stronger position than most.
Stay current on your favourite topics
These are six core challenges we see financial services firms being forced to think about when starting to adopt PaaS services in the public cloud and we’ll spend subsequent blogs going into more detail on these topics. PaaS is proving to be worth the investment, but broader adoption in financial services requires innovation across the industry with both CSPs and clients having to rethink and rapidly mature solutions and operating models on multiple fronts.