Maeña Twomey, programme manager, Lloyds Banking Group, looks at the issue of enhancing assurance on large cross divisional change programmes

The growing trend for larger, more complex change programmes calls for an elegant approach to test subsequent target operating model impacts. Live trialling is a way to do this by enhancing standard testing to prove the operating model post-implementation in a low risk manner

Traditional testing assurance

It is widely acknowledged in change programmes that issues are exponentially more expensive to address the later in the life cycle they are found. None are more costly than those problems found post-implementation. While this principle commonly refers to system defects the same applies to operating model issues. Operating model issues include SLAs, team-to-team interfaces, organisational flow of data and service and critically customer experience.

Traditionally organisations will replicate production, as far as possible, by creating a pre-production or test environment. This aims to prove the technology and usability. However, a production environment has uniquely ‘evolved’ and therefore only in rare/costly circumstances can you fully replicate this. In addition, what is generally overlooked is the requirement to prove any new processes end-to-end and therefore the target operating model.

Traditional testing assurance:
• Gather requirements
• Design and agree the solution
• Build the technology
• Test the solution
• Train and communicate to the users
• Roll-out

Shortcomings in the traditional model

While widely used, this model has shortcomings when it comes to large, cross divisional change programmes where there are a significant amount of cross department dependencies.

Key shortcomings are:
• Limited proving of inter/cross divisional hand offs
• Limited proving of SLAs (both internally and externally with third party suppliers)
• Limited proving user and customer experiences
• Limited proving all of the above in a production environment - certain aspects of a production environment are inherently unique e.g. reference data and batch processes.

Issues resulting from these are generally customer facing and therefore have an impact on an organisation’s reputation and worst case scenario its regulatory compliance. Take, for example, retail banking account opening for a new product. Looking at the components involved in this:

Customer expectation:
• Open an account via telephony, online banking or in branch.
• Account opened within agreed timeline
• Card, cheque book, welcome letter, PIN etc issued

Now look at this same process from an operational perspective:
• Open account is initiated via different systems\routes and passed to a single back office team
• This team must then deal with the opening of this account and pass it onto additional teams e.g. fraud
• This account is opened and then internally new letters are printed and issued
• At the same time data is sent to an external third party to produce the card
• Within this process you have numerous internal and external dependencies. Teams are all dealing with material (the product) and data which must be seamlessly passed, worked upon and understood all the way through the process.

This is a complex array of change, all of which is customer facing. In order to be successful there is one major dependency, all departments and third parties have worked seamlessly throughout the lifecycle and understood all requirements exactly. As we know, this is simply not the case, hence the need for traditional testing assurance. However, this model falls short when proving customer and operational aspects end-to-end.

A better way - live trialling overview

Standard project lifecycle exists because we will always find issues with our design and build of technology and operating models. The traditional project lifecycle and associated testing assurance is not entirely aligned to addressing issues realised when the technology and operating model are brought together in production and the level of change is significant.

So what can we do to address this? The answer is to build on a simple idea commonly used in small single change projects - the pilot. Rather than simply pilot a system in a ring-fenced simulated manner, look to extend this concept to include a customer and user experience in the real world.

Look not at the minutia of the detail, for example, the screen layout etc, look instead at the bigger picture, treat the systems as black box. Operate in a controlled post-implementation situation and monitor this, i.e. carry out live trialling.

Implementing live trialling - live trialling principles:
• Your production environment is inherently unique and simply cannot be exhaustively replicated:
• All traditional testing assurance (system test, UAT etc) is completed;
• All traditional implementation proving is complete e.g. training;
• Your live trialling is outcome focused - you identify key outcomes of a process and prove these. The key assumption is that all underlying functionality is stable.

Live trialling approach:

Step 1 - Maintain the status quo
• Deliver your change in line with the traditional testing assurance model
• Fully test and implement your change

Step 2 - Know your ‘moments of truth’
• Identify your critical business processes, inter divisional hand offs and corresponding SLAs. Use your frontline and business subject matter experts to identify these
• Identify the outcomes you wish to prove to gain assurance and the checkpoints in the end-to-end process

Step 3 - Manage the risk
• Create the ability in your production environment to operate the system end-to-end in a ring-fenced manner (e.g. in a retail bank, create a new sort code against which customer accounts can be created and easily managed; alternatively creating a data format which you can easily manage)
• Confirm the readiness and support of your third parties in doing this.

Step 4 - Start small and build up
• Use the real users and not testers to carry out the proving
• Carry out your business processes in a BAU manner
• Start with small and simple levels of assurance and increase the breadth and complexity as your assurance increases

Step 5 - Monitor the outcomes
• Monitor the outcomes in three areas: Customer and user experiences, technology outcomes.

In summary

While there is no substitute for traditional testing in carrying out live trialling you will have realised extra assurance for:
• The integrity of your target operating model design
• Your cross divisional hand off and SLAs
• The integrity of your production environment when new data is introduced
• Your colleagues’ readiness to accept this change
• The readiness of your third parties to meet your requirements

Live trialling is inherently simple, low cost and low risk to implement as you leverage existing implementation activities to deliver this. This phase will bridge the gap between traditional change assurance and what is required for large cross divisional change programmes.

Share Story: