In today’s fast-paced technological environment, major applications require regular upgrades every few years. These upgrades, while critical, can be highly time-consuming for IT departments and challenging to justify to business units, particularly if the immediate business value isn’t clear. This often results in banks prioritizing other projects over upgrades, which can lead to project delays and frequent incidents on production platforms.
This article draws insights from our experience with the Temenos Transact upgrade, though many of the points discussed are applicable to other core banking systems as well. Understanding these challenges can provide valuable lessons.
Pre-requisite before starting the project.
Pre-requisite before starting the project:
Check the Stack.
Check the Technology Stack: Thoroughly review the versions of all key technologies involved in your platform. This includes verifying the Java version, JBoss version, operating system (OS) version, and SQL Server version. Each component must be compatible with the new platform you plan to implement.
Identify and Address Discrepancies: If any discrepancies are found between the current technology stack and the requirements of the new platform, these must be carefully noted. It’s important to budget for necessary updates or changes, especially significant ones like operating system upgrades.
Consider the Impact on Timeline: Updating critical systems such as the operating system can have a significant impact on your project timeline. It’s important to account for this in your planning phases to avoid unexpected delays and ensure that timelines are realistic and manageable.
Review technical elements in Used.
Develop a tool to analyze menu composition and identify the processes in use. If possible, remove unused code from the repository. This will allow a focus on frequently used elements and avoid unnecessary code conversion. In other words, it will reduce the scope and avoid additional costs and maintenance
During project:
Try the runbook as many times as possible.
Review the Runbook: Start by verifying that the runbook for the IT team includes all necessary steps for the upgrade process. Ensure that every detail, from initiation to completion, is documented clearly and comprehensively.
Rehearse the Runbook Extensively: Conduct multiple rehearsals of the runbook to familiarize the team with every procedure. This practice not only helps in identifying any missing steps or inconsistencies but also enhances the team’s ability to execute the upgrade smoothly.
Time the Rehearsals: It’s important to time these rehearsals to understand how long each step takes. This information is crucial for planning the upgrade, especially if it needs to be done over a weekend. Timing helps in setting realistic schedules and ensures that all stakeholders are aware of the expected downtime or limited availability.
Multiple Executions: Performing the run through just once or twice may not be sufficient. Repeat the process as many times as needed until the team feels confident in their ability to execute flawlessly. This repetition is key to a safe go-live weekend, reducing the likelihood of unexpected issues and ensuring a quick response to any problems that arise.
Checks Technical Logs.
Compare Logs Before and After the Upgrade: Carefully review JBoss logs, SQL logs, and application logs from before and after the upgrade. This comparison is important for identifying any new errors or unusual behaviours that emerge as a result of the upgrade.
Control and Understand New Errors: Ensure that all new errors introduced by the upgrade are thoroughly understood and under control. Investigate the source of these errors to determine whether they are critical or can be addressed in a less immediate fashion.
Categorize and Prioritize Errors: Classify new errors related to the update based on their severity and impact on system operations. For less critical errors that do not disrupt primary functions, scheduling fixes post-go-live can be a viable approach. However, more severe errors that affect core functionalities or data integrity should be prioritized for immediate resolution.
Check parameters has not been overwritten.
Upgrades usually come with the latest functionality, and in some cases, it could create regression for local set up. For example, your swift release could deviate from standard due to counterparty requirements. By installing the upgrade, you could lose this local set up if you do not run a full parameter comparison.
Compare Database via a tool: Extract all parameters from both databases and run a compare tool.
The upgrade process must add parameters but not overwrite existing parameters.
Get the latest update.
Make sure the latest releases are installed for all the modules in your product. As you upgrade, better have the latest software version and include all updates / patching done by the software company.
Make sure no pending are left after running upgrade process.
This must be part of runbook, but please ensure that no records are left unauthorised after running the upgrade process.
Install 2 DB same system day and preferably end of quarter, one without performing the upgrade.
Install two databases on the same system, ideally at the end of the quarter—one that has undergone the upgrade and one that hasn’t. This setup is key to proactively identifying issues in complex processes such as legal reporting, portfolio valuation, and fees calculation by allowing direct comparison at a critical financial period.
For systems like Transact, it’s important to monitor the close of business (COB) times before and after the upgrade. Any significant changes in process duration should be documented and analyzed to ensure efficiency is maintained.
Ensure there are no mismatches or differences in the bank’s general ledger by comparing the details before and after the upgrade. If the bank book contains fewer than 200,000 positions, Excel can be used for this comparison; otherwise, a more robust solution is necessary.
For interface files, including those produced in XML format, use tools like Notepad++ or an in-house solution to ensure identical content pre- and post-upgrade. This is crucial when documents such as letters or reports are generated through these files.
Also, check the consistency of portfolio valuations and fee calculations. Figures should match between the two databases, and any discrepancies should be thoroughly analyzed. Similarly, ensure that the figures in legal reporting files remain consistent. Discrepancies in these critical areas must be addressed promptly to maintain the integrity of financial and legal reporting.
By following these practices, you can ensure a seamless transition during platform upgrades and maintain the reliability of your data.
Testing.
Engage Business Users: Ensure that representatives from relevant business units are involved in the testing process. Their firsthand experience with the system’s functionalities makes them invaluable in identifying issues that may not be apparent to the IT team.
Comprehensive Functional Testing: Business users should test all aspects of the software to ensure everything operates correctly under real-world conditions. This includes workflows, reports, user interfaces, and any integrations with other systems.
Scope of This Article: It’s important to note that while this article acknowledges the significance of involving business users in testing, it does not go into the detailed methodologies of conducting such tests. The focus here is on the broader aspects of preparing for and executing a platform upgrade.
After go live:
Get ready for hypercare period.
Initiate a Hypercare Period: Collaborate with your provider to set up a hypercare period. This is a critical time when support is intensified to address the potential increase in incidents that often accompany major IT changes.
Anticipate and Manage Incidents: Be aware that incidents are likely to increase following the upgrade. This can be due to the new system configurations, unfamiliar interfaces, or underlying issues not identified during testing.
Scale Your Response Capabilities: As a manager, it’s essential to scale your team’s capacity to handle incoming queries and issues. Ensure that sufficient resources, including personnel and technical tools, are available to manage and resolve these incidents efficiently
Prepare for Business Impact: Equally important is preparing to address any business impacts. Communicate with relevant stakeholders about potential disruptions and set expectations for issue resolution timelines.
Don t forget to update your cybersecurity document.
Check Folder Structure: Review the folder structures before and after the upgrade to identify any changes. Analyze these differences and update your risk documentation if there are any impacts on your security protocols.
User Rights Comparison: Compare user rights before and after the upgrade. Ensure all changes are documented, especially any new rights that are required post-upgrade. This helps in maintaining the integrity and security of user access and control.
Technical User Documentation: Document any new technical users who are needed during this upgrade. Include details such as their roles, rights, and the duration for which they will need access. This documentation is critical for audit purposes and maintaining security standards.
Update TAL: Modify your Technical Architecture Layout (TAL) to reflect new server names and any new components added during the upgrade. This ensures that your architecture documentation remains accurate and up-to-date, which is essential for both ongoing maintenance and security compliance.