Future Proof – Protecting our digital future

photo credit: Rusty Clark

Organisations run on information. Without good digital business information, organisations will be hampered by radically increased business risks, liabilities and inefficiencies.

In even the most general scenarios, managing your digital business information so that it is accurate, accessible and accountable is a challenge. But seeking to manage it successfully through any of the following seven scenarios raises its own specific challenges. In no particular order, here’s a quick discussion on why, and some ideas for what you can do to combat these particular digital recordkeeping challenges in your own organisation.

Moment number 1 – System migration

System migration is a key point of risk for digital information.

A large amount of intentional and unintentional data loss can happen during migration. In any system about to undergo migration, it is important to identify what records need to be carried forward out of the system, what their metadata dependencies are and how they will be able to function in their new target system. Unless metadata requirements in particular are considered, migration can be a significant point where records and the metadata supporting their meaning and value can be irrevocably lost.

State Records’ Managing digital records guideline has lots of advice about how to manage migration projects to ensure that accurate, meaningful and useable records can be maintained.

Moment number 2 – When systems that are integrated are upgraded or changed

Many organisations have developed integration pathways between their business systems and their EDRMS environments. These enable records to be easily exported out of the business system and into another application that will take care of their long term management.

These pathways usually work well and integration can be a very good recordkeeping solution, but integration pathways do need to be redefined and reassessed whenever either or both systems are upgraded.
The following is an example of the types of small but significant changes caused by the upgrade of integrated systems that can affect good recordkeeping.

Example:

A council uses a business system to accept, process and respond to customer enquiries, request and applications.  Because the system connects customers with all parts of the Council’s operations, it has become a key business system in the Council.

The Council has integrated the system with its corporate EDRMS. As the system operates according to designated workflows, these have been designed to automatically export records into the EDRMS at specific points. These records are saved into the EDRMS with good metadata about the business process, the internal action officers, the customer and the dates and times of all actions.

In the last couple of years, upgrades for the business system have been offered every 18 months. Business staff need these upgrades because they offer new and useful functionality. After the most recent upgrade records staff assessed the records automatically exported to the EDRMS to see if they were still capturing full and accurate information about the transactions performed in the business system.

They found that the upgrade had affected the system’s ability to import agent metadata into the EDRMS. Previously the integration pathway had automatically loaded the names of action officers who dealt with clients or made decisions or who created records, directly into the relevant fields in the EDRMS. Now, post upgrade, the system is instead exporting the default metadata value ‘CBC server’ and not the name of the person into each user field, including audit data fields about management actions performed in the system.

The Council has business, reporting and accountability needs to have the details of action officers who are responding to requests and creating records. Not having this metadata is a risk to the effectiveness and accuracy of key business records. They are currently negotiating with the vendor to rectify this situation, but the example shows how important it is to monitor business systems and assess their ongoing accuracy.

Moment number 3 – When you are configuring systems

System configuration and implementation is a critical recordkeeping pressure point. Obviously how you design, establish and roll out a system has a major impact its recordkeeping capacities. When configuring your systems you need to check whether all relevant metadata fields are enabled, whether all appropriate functionality is in place and whether the types of records you want can actually be generated by the system.

Also, in terms of information longevity and sustaining business information for as long as you actually need it, documenting your system configuration is quite critical to long term sustainability.

For example, one large government organisation used SharePoint in all its business areas. In the 3 years that they ran the system, the organisation reconfigured and readjusted their SharePoint implementation many, many times. These configurations were rarely documented. When it came time to migrate the system, there was no accessible, usable documentation to accurately describe how the system was being used. The migration was therefore a very laborious and time consuming process. There was also no way of knowing what the most important components of the system were or whether the metadata that was extracted and migrated with the records was the most valid and appropriate metadata record. Metadata is frequently overlooked, but how it is configured is key business information. Without a record of how its metadata has been applied, a business system is much harder to maintain and manage.

Similarly in web 2.0 applications such as blogs, good documentation of the system architecture you implement is needed to migrate and sustain these environments and their data through time, and through the frequent software upgrades that affect these applications.

Moment number 4 – When you are using unusual, aging or complex formats

Unusual formats, aging formats and complex formats can be difficult to sustain through time. Managing these formats through time is a key recordkeeping risk.

A January 2012 article in Stars and Stripes, the US Military newspaper, ‘As technology evolves, military wrestles with preserving vital engineering data’, providing fascinating insights into the complexities of the problems and outlined the considerable work that is going into determining solutions.

The article outlines the problems the US Defense Department is having with preserving its electronic data for the long term. The key problems they are facing are format related and relate to keeping data accessible and trying to ensure that data is free of inconsistencies.

The problems have evolved because Defense assets, such as ships and planes, have a very long life span. However, the formats that their plans and blueprints are in are complex digital formats. Mark Conrad, an electronic data specialist for the National Archives and Records Administration is quoted in the article as saying, ‘When the Navy deploys a new ship, they expect it to be around for 50 years, but the CAD software used to produce it has a lifespan of about 18 months. Software companies make money by introducing new bells and whistles…but when you try moving the information from one system to another when you upgrade software, system A’s bells and whistles may not be supported by system B’. Therefore asset plans can need to be migrated as frequently as every 18 months.

Mistranslations can occur when you are trying to upgrade or migrate and bring old formats up to newer versions. A senior consultant from a company specialising in solving CAD interoperability issues says that these problems ‘Can happen all too quickly when you transfer engineering data from one system to the next. There’s either data loss or data is interpreted in ways it wasn’t originally intended.’ ‘What’s really got people concerned in the subtle problems you don’t notice that creep in, and they assume the model data is correct. It’s only when you get out of the digital world and into the real world that you realise you’ve got serious problems.’

Therefore, if you need to access, keep or use your digital information for periods longer than a 3 to 5 year window, pay careful attention to your choice and management of formats. And if your records in complex format support long term assets such as buildings, infrastructure or critical equipment, pay very careful attention to the management of these records and be alert to any subtle changes that may occur to the records through time. Do what you can to mitigate and manage any vulnerabilities.

Moment number 5 – When you are considering the use of cloud systems

Moving corporate information to the cloud poses a key risk.

For any cloud strategy, you need to ensure that the quality and trustworthiness of your records are not compromised in the cloud environment. You also need to ensure that there are good data export capacities in place. Where appropriate, you need to be able to bring good records and good metadata in accessible formats back into your organisation at the end of your cloud arrangements.

It does need to be emphasised that use of the cloud by NSW government agencies is only legally possible if agencies have:

• assessed and addressed the risks involved in taking and sending records out of the State for storage with or maintenance by service providers based outside of NSW.
• ensured the service providers facilities and services conform to requirements in standards issued by State Records
• ensured contractual arrangements and controls are in place to ensure the safe custody and proper preservation of records
• ensured that the ownership of the records remains with the public office, and
• ensured that they monitor the arrangement to ensure the service provider is meeting relevant requirements.

As part of assessing the risk, all relevant statutory obligations should be assessed, including the special restrictions on the disclosure of information outside of NSW in s.19 of the Privacy and Personal Information Protection Act 1998. Failing to comprehensively assess risk would be a breach of section 21 the State Records Act, and possibly other legislation as well.

Monitoring and regulating cloud service arrangements can be challenging, because increasingly it seems that the move to cloud systems is happening at business unit level, without IT or records involvement and often without formal risk assessments and approvals. All managers and staff need to be made aware of the risks involved in cloud strategies, in terms of the ongoing use and long term maintenance of valuable business information.

Moment number 6 – When your information needs to be kept for a long time

Maintaining records for as long as they need to be kept is one of the fundamental challenges of digital recordkeeping.

Your organisation will have business, legal, accountability, evidential and informational needs to have certain records for long periods of time. Quite a large proportion of corporate records will need to be kept for 5 – 10 years. A significant proportion, relating to employees rights and entitlements, key areas of business operations and risk, corporate assets and a range of other issues will have legal retention requirements that extend beyond 50+ years.

Keeping accessible, useable and meaningful digital information for these types of time frames is one of the key risk and accountability issues for all organisations.

It is therefore important to identify now which corporate digital information is needed by your organisation for periods beyond 10+ years. Try to flag this information as it is created. Try to create it in standard, widely used formats. Protect it during migration. Do not allow it to be sidelined in a legacy system or offline storage arrangement. Monitor it and ensure the metadata that sustains its meaning and accountability can be kept and kept connected to relevant records. Proactive approaches such as these are much more cost and risk effective than attempting to real retrospectively with the loss of long term, critical business information assets.

Moment number 7 – When lots of people start using mobile devices

There is increasing use of mobile devices across Government. This growth is being triggered by:
• the ubiquity and ease of use of these devices
• a more mobile workforce requiring more mobile technology
• the normalisation of the ‘as-a-service’ business model that is pushing many applications and their data to the cloud
• the productivity and communications benefits that mobile technologies enable, and
• the growing move to ‘BYOC’ (bring your own computer) models which require less initial ICT cost and maintenance.

Initially mobile devices were only used to view existing content. Increasingly, however, they are being used to create and manage business records. In most organisations it tends to be high level staff using mobile devices and so they are being used to transact high level business operations.

Mobile devices already contain unique copies of corporate data and this is a trend that is only going to increase. The variety of devices in common use is increasing rapidly, as is the diversity of operating systems and versions of operating systems they run on. Also increasing is the variety of communication methods (email, Webmail, SMS text, instant messaging, social media, etc.) supported by mobile devices. All of this potentially adds complexity to recordkeeping.

If use of mobile devices for business purposes is underway in your organisation, ensure that there are good corporate policy statements in place that require staff to export business records out of their mobile devices and into the relevant corporate systems. Various ICT solutions also exist that enable users to operate within the corporate network while using mobile devices. For high risk business operations and key business staff, investment in these types of infrastructures may be prudent to ensure all relevant corporate information remains within business environments where it can be managed, protected and used.

Do you have any other suggestions for moments where you need to think ‘Is my digital information OK??’. Can we make this list a top 10? We would love to hear your ideas, so please let us know what you think!