Decommissioning business systems – lessons learned on the journey so far December 23, 2015
We have been talking with a range of NSW public offices about their experiences in decommissioning systems.
One of these organisations has a range of different business systems running on Windows 2003 servers, so has been involved in a number of decommissioning projects this year. Below they have very generously summarised and shared some of the lessons they have learned.
Please note, this organisation is not presenting any of their advice as definitive:
We have been on a steep learning curve and our projects have involved a lot guesswork! We would welcome feedback through Future Proof from others on how you have done things better or differently, but for better or worse, these are the decommissioning decisions we have made this year and some of the lessons we have learned in the process.
1. Work closely with IT colleagues
IT colleagues are primarily responsible for decommissioning projects. If you can build a good relationship with IT and develop a good process to support their decommissioning work and timelines, you and your IT colleagues can work really successfully together to decommission legacy systems, maintain the corporate information that needs to be preserved and authorise the destruction of time-expired information. Our experience is that IT colleagues are really quite keen for records and information management advice during decommissioning projects. When a system is decommissioned, it’s effectively turned off. It is a big decision to unplug a corporate application and all the information it holds and so we have found that IT is quite happy to leave the information related assessments and recommendations to records and information management staff, while they handle all the IT complexity.
2. Get records and information management onto decommissioning project management checklists
Project management governs so many business and IT priorities. If you can, get records and information management issues incorporated into project management methodologies. On decommissioning project management plans, a simple project step ‘Consult with records and information management’, can help to ensure that necessary records are maintained through any decommissioning project.
3. Business staff really know where it’s at
When you are working on a decommissioning project, business staff who know the system are your most valuable asset. Business colleagues will help you to identify what information needs to be maintained through your decommissioning project. Fundamentally, they are the ones who know the information required to support business processes, know what information is needed in response to different enquiries, and know what information will be required in the long term to meet business needs.
4. Retention and disposal authorities are always your friends
Retention and disposal authorities are just as useful in system decommissioning projects as they are in traditional sentencing projects. They still give you the broad rules to apply, you just apply them in a different framework.
5. Be aware of significant upcoming obsolescences
In our organisation, and many others, 2015 was a big year of Windows 2003 server decommissioning. This was because Microsoft stopped support for all its 2003 servers late this year. This was a big incentive for all organisations using 2003 servers to move away from these environments and onto newer, supported environments. Our organisation did not have any active systems on 2003 servers but had a number of legacy systems that were being maintained for reference purposes on the old servers. The systems had already been replaced in the business environment by newer technology but the legacy information in the old system still had ongoing reference and reporting value and needed to be assessed to determine if it should be exported and moved to a new management environment.
So it’s useful to know the major platforms, server architectures and other major infrastructure dependencies that your organisation has. There is so much movement in this space, knowing what major transitions might be coming up will help you to do a bit of forward planning and help you to understand what decommissioning projects might be coming your way, or buy you some time to start lobbying for inclusion in some big upcoming decommissioning projects. For us, the big upcoming obsolescences I think involve the cloud. As with most organisations, lots of infrastructure has already moved to the cloud, so knowing what has moved and when it is likely to need to transition to a new service environment represents the future of many decommissioning projects.
6. Metadata, metadata, metadata – it’s important and it’s tricky!
As always, metadata is a biggie. It’s huge, it’s everywhere, it’s integral to the meaning and accessibility of information, it’s high and low level, it’s frequently incomprehensible to all but those who use it on a daily basis and sometimes it’s proprietary and can’t actually be extracted from the system it’s part of. So metadata can be a tricky one to deal with.
Business colleagues were vitally important in defining what metadata really contributed to record integrity, accessibility and useability. We didn’t define it like that but instead asked questions like ‘how do you do a search for records about…?’ or ‘how would you know this is an accurate record of…?’ Talking to them about how they search for or use records from the legacy environment helped to define what metadata needed to be exported with records to make them trustworthy and useful.
Business was also critical for helping us to interpret metadata and to expand acronyms on field names into meaningful values.
Some metadata we couldn’t extract. In one system, the audit trail metadata that documented who updated which data field and when was stored in a proprietary format that needed the original application and data dictionary in order to be read. The application and the data dictionary were being turned off and left behind, so essentially this metadata had to be as well. The business told us that this audit trail metadata was infrequently referenced for 2 – 3 years after the data was first created but now, 15 years down the track, they never needed to reference this data, even when information from the system was requested for audit or legal enquiries.
GA28 and GA39 both say that ‘System logs may be required for accountability purposes or as evidence in investigations to trace who accessed what records. The length of retention will be dependent on the organisation, the system and the nature of the risks faced. Retain in accordance with the organisation’s requirements, then destroy’. So after talking with the business and knowing the technological problems with data export, we documented these issues and made the disposal recommendation that this audit metadata was no longer required and could therefore be destroyed with the system.
7. Build for what we are trying to achieve – a useable, meaningful record
As mentioned above, all the systems we have been decommissioning are legacy systems that were replaced with new in the business a number of years ago. As a result, the decisions we had to make about what type of records we needed to generate through decommissioning were pretty limited and simple. We really only had the capacity to build reports and the most effective means of managing those reports in the long term was to export and then store them in our EDRMS.
We did work with the business however to choose the records we wanted. As discussed, we worked with the business to identify the system data that represented records of ongoing value to the organisation and designed reports to run this data to CSV.
We left lots of data behind. Our disposal authorisations list the data we appraised as no longer having business value for the organisation. Lots of system components weren’t actively used, or only partially used, or only ran very short term value reports etc. In one system we only extracted two reports (both interestingly relating to legacy financial issues, because there were still occasional customer enquiries about this data) – all other data had effectively been carried over to the new system several years previously.
In our report construction and EDRMS folder design, we have tried to build for maximum usability and accessibility. The records we have extracted have long term retention requirements so they will need to be understood and referenced for a long time into the future. For one key system that we needed to export a lot of data from, we worked with the business to create a master index to all the reports, to try and create a central register that will enable better and easier information accessibility. The folder structure in our EDRMS also gives information about this business system at the top level and then links down to the master index and the specific reports, each of which is named to give the system, date and business context to help explain what these records are.
We are also still finalising a lot of documentation to explain each of the decommissioning projects, outline the decisions we made and to explain how the data that is now serving as our record of these system was generated. Our IT colleagues are using this documentation as part of their project gateways and sign offs.
8. The State Records Digital Archives Migration Methodology has been really helpful
All through our projects this year we have referenced different components of the DA migration methodology process and documentation. They have been really beneficial in helping us to consider and try to build accountability around our decommissioning processes. With our decommissioning projects, we are fundamentally wrenching systems apart, pulling out tiny informational fragments and trying to make this stand as evidence of the processes that were represented in a much larger and more dynamic whole. The DA methodology has helped us to realise how much this process of pulling a system apart to understand its informational core needs to be documented, explained and agreed to, if the information you are extracting is going to be able to stand as valid and accountable evidence of the system it was once part of.
9. These projects take time
No one can do a decommissioning project by themselves. It therefore takes a lot of time to get the right people on board who can all resolve their pieces of the puzzle. Business areas require a heavy involvement and it can be hard for them to find the time for legacy system assessment. They are fundamental partners in this process though and so it may take longer than you think to meet and address all the questions you have for them.
10. We can only do our best, share our knowledge, learn as we go and look to the future
As discussed at the start, we are new to these types of decommissioning projects that are all about data and metadata in business applications, not about documents in a network environment or EDRMS. It’s a different form of record to pull apart and understand but basically it’s still all always about who, what, when, where and why. It’s about identifying the best information that tells us what the business was, when it happened and what the outcome was.
In our projects we gave it our best shot, talked to everyone we could, thought about use and accessibility and we’re trying to document it all as best we can. When the dust settles we’ll look back and assess and keep trying to improve our processes.
As next steps for the future decommissioning projects we’ll be involved with, we’d like to try and develop some checklists to streamline our processes and also to engage more with project management staff. We also want to try and do as much as we can ‘by design’ with new systems. All new systems will be decommissioned systems one day, so by having conversations up front about information management in new systems or services and by getting good documentation and contract agreements in place, we’ll hopefully make both migration and decommissioning easier processes down the track.
” Work closely with IT colleagues” Lol!