On 5 February 2021, the Financial Conduct Authority (FCA) published a report titled ‘Implementing Technology Change’, which sets out findings from its review into how financial services firms manage technology change, the impact of change failures, and industry practices to help reduce the impact of incidents resulting from change management.
The FCA analysed over one million production changes implemented by a sample of firms in 2019, and supplemented its data with a qualitative questionnaire, a confidential board questionnaire, and industry workshops. The report – which should be read against the backdrop of several high-profile information technology (IT) failures over the past decade and ongoing technological advancements in the financial services industry – reveals that failed technology changes are one of the main causes for operational disruption within firms, accounting for a quarter of all high severity incidents that cause harm to consumers and the market.
The report is relevant for all financial services organisations and will be of interest to technology providers. The FCA’s findings contribute to discussions surrounding Operational Resilience (see our latest article on the topic here) and how firms can implement changes to their technology in ways that reduce the potential for operational disruption.
Below is an overview of the FCA’s report.
Good practices and bad practices
The FCA began its report by outlining the practices identified as contributing to technology change success and failure. Starting on a positive note, the FCA said that – while no single, perfect approach exists – stronger governance, day-to-day risk management, increased automation, and more robust testing and planning can contribute to successful change and less disruption. The FCA emphasised the following take-aways:
- firms with well-established governance arrangements have a higher change success rate;
- relying on high levels of legacy technology is linked to more failed and emergency changes;
- firms that allocated a higher proportion of their technology budget to change experienced fewer change-related incidents;
- frequent releases and agile delivery can help firms to reduce the likelihood and impact of change-related incidents; and
- effective risk management is an important component of effective change management capabilities.
In terms of bad practices…
| Practices identified as contributing to change failure | Possible solutions |
| Lack of complete visibility of third-party changes. | Workshop participants suggested that firms should ensure that third-party contracts are clear on how changes are communicated and the potential impacts to a client firm’s IT estate. |
| Change management processes that are heavily reliant on manual review and actions. | Firms could consider the benefits of repeatability and consistency that automation can bring, whilst also recognising the drawbacks. |
| Existence of legacy technology, which impacts firms’ ability to innovate. | Firms could consider adopting more innovative approaches – through solutions such as DevOps, micro-architecture and the public cloud (discussed below) – although migrating away from legacy technology brings its own risks. |
| ‘Major changes’ – such changes were twice as likely to result in an incident when compared with standard changes, which workshop participants attributed to the complexity of such changes and the inability to break them down into smaller components. | Change Advisory Boards (CABs) were used by some firms as an assurance mechanism. However, the FCA questioned their effectiveness in some instances, as CABs approved over 90% of the major changes reviewed by the FCA in 2019, and in some firms the CAB had not rejected a single change. Workshop participants suggested that CAB members should be carefully selected to ensure that the requested changes are thoroughly checked. |
Incidents caused by technology change
The FCA looked at the impact of incidents caused by changes to firms’ technology. Generally, technology change was managed effectively, with only 1.6% of changes resulting in an incident in 2019. However, this still represented over 13,767 incidents, 14% of which had customer-facing impact. In particular, ‘major changes’ were more likely to result in incidents. Other notable types of incident included:
- customer-facing and high severity incidents, regarding which workshop participants highlighted the importance of having in place comprehensive and well-tested roll back plans to minimise the impact on customers, for example internal and external communications explaining what alternative channels are available to customers;
- incidents resulting from emergency changes, which can be time-sensitive and without the usual rigorous assurance and governance. In this context, workshop participants stressed the importance of Subject Matter Expert (SME) reviews, good governance, and having in place the right culture; and
- incidents resulting from third party changes.
Importantly, board members said that firms should learn any lessons from failed changes.
The change process: Governance, management, build and deployment
The FCA’s report went on to discuss how firms govern, manage, build and deploy technology change. Today, these changes are being driven by – among other things – more demanding expectations by increasingly sophisticated customers, new regulatory requirements, and developments in relation to Covid-19, data protection and the UK’s exit from the EU. On change budget allocation, the FCA found that firms dedicated the highest proportion of resources to ‘maintenance and upkeep’ (32%) - greater automation of which promises to allow firms to re-allocate resources to other change activities and to ‘satisfying regulatory and legal requirements’ (21%).
From a governance perspective, the FCA found a positive correlation between the longevity of change governance arrangements and higher change success rates. Best practice includes reviewing governance arrangements periodically and ad hoc, and encouraging SMEs and Non-Executive Directors (NEDs) to constructively challenge governance. Most firms also used a consolidated tool to track, categorise and record changes made across their estate. Looking ahead, automation is high on firms’ agenda. From a project management perspective, the FCA referred to two broad types of methodologies used by firms, ‘agile’ and ‘waterfall’ methodologies, and found that firms using a higher proportion of agile methodologies had fewer incidents resulting from change (although in practice firms often use a hybrid approach).
Firms agreed that the key risk factor prevalent in high risk change projects is dependency on other projects creating the need to coordinate and keep track of a moving puzzle. On mitigating possible issues, the FCA’s report emphasised the role of testing, and noted that firms remained reliant on manual testing and peer review, as opposed to automated alternatives. In relation to the build and deployment stages, the FCA first assessed the benefits and drawbacks of a DevOps model, which integrates traditionally siloed development and operations teams (and quality assurance and security teams in some cases). The FCA then focused on the speed of technology change, noting the use of automated tools and processes, modern infrastructure, and public cloud solutions by some firms.
From legacy infrastructure to the public cloud…?
Finally, the FCA’s report looked at how infrastructure impacts technology change. Over 90% of reviewed firms were still reliant on legacy infrastructure and applications to deliver production services. Moving away from legacy infrastructure, however, presents challenges; indeed, several of the highest impact technology incidents over the past decade have been a result of failed migration of legacy infrastructure. Public cloud solutions – which have now been around for some time – continue to provide new opportunities, such as enabling a higher degree of automation, reducing the manual risks of change, and increasing the agility of incident response. However, the challenges that come with cloud solutions should not be ignored, particularly with regard to the potential lack of oversight and direct control.
Stay tuned for Herbert Smith Freehills’ new Operational Resilience hub, which will launch on 24 February 2021.
Key contacts
Disclaimer
The articles published on this website, current at the dates of publication set out above, are for reference purposes only. They do not constitute legal advice and should not be relied upon as such. Specific legal advice about your specific circumstances should always be sought separately before taking any action.