Future Proof – Protecting our digital future

For Information Awareness Month here at State Records, we are doing a series of posts on key challenges facing information professionals today.

And, Master Chef style, these challenges don’t get much bigger than digital information retention and disposal.

The ad hoc and unregulated deletion of information and the unwarranted over-retention of information can both critically affect business performance. Therefore, decisions about how long to keep information are strategically fundamentally important.

State Records NSW issues authorised rules that outline how long all forms of government information legally need to be kept.

But how can these rules best be applied and what are the risks if they are not?

Information retention and disposal rules must be deployed ‘by design’

As most government business today is performed within digital systems, applications and services, it is vital that information retention and disposal rules are considered at:

• system design
• system procurement
• system implementation
• transitions to cloud services
• contract negotiations for cloud services
• portability planning for cloud services
• business process outsourcing
• application development

If, at each of these stages, proactive planning for the authorised retention and disposal of government business information can be incorporated ‘by design’, significant costs and business risks to government can be mitigated.

Business risks that occur if digital information retention and disposal are not addressed

Risk: Increased and unsustainable storage costs

ICT research company IDC has predicted that annual world-wide growth in data volumes is currently 60% per year.

All organisations face significant increases to storage costs as a consequence of rapidly increasing data volumes.

Basing his estimates on the IDC’s data growth rates, the decreasing bit density on the platters of disk drives and the 0-2% current annual growth rates of ICT budgets, Stanford University computer scientist Dr. David Rosenthal has observed that:

10 years from now, storing all the accumulated data would cost over 20 times as much as it does this year. If storage is 5% of your IT budget this year, in 10 years it will be more than 100% of your budget.

Risk: Increased and unsustainable management costs

The costs of digital storage containers are decreasing but the costs of the storage software necessary to access, govern and manage business information are rising rapidly.

Keeping more information will also necessitate more information management, which will again increase costs.

Risk: Loss of high value information in amongst the ‘noise’

Numerous sources have reported that issues around document profusion are already impacting their business environments.

For example, organisations are often uncertain about which is the final or official version of a document, with the result that everything is kept and nothing gets destroyed.

Good governance, strong processes and sound change management are therefore core to establishing good and trustworthy disposal practices.

Risk: Increased risk of inadvertent data loss through large-scale data purging

Poorly designed systems or processes will result in large volumes of organisational data that are not aligned or mapped to their relevant business requirements or retention rules.

As storage, licensing, migration, contract or service costs become too large, large-scale data purging will be utilised as a cost minimization option in some environments.

This option however risks the inadvertent loss of high value business information in the purging process, potentially resulting in negative legal, client, service, audit, accountability and/or business consequences.

Risk: Increased costs through a ‘keep everything’ approach

By recognising the risks of inadvertent disposal but by not adopting an appropriate approach to its management, some organisations are resolving to keep all their digital information in perpetuity. The risks that can result from this approach are:

• unsustainable storage costs
• ongoing licensing costs for systems holding legacy data
• slower or ineffective information retrieval
• no prioritization and management of high risk/high value information
• extensive digital continuity management issues that will become unsustainable in the medium to long term

Risk: Poorly managed and costly services

If information retention and destruction requirements are not identified and understood before services are provisioned, potential consequences are:

• unnecessary ongoing costs to retain information that is authorised for destruction
• arrangements for required data portability are not identified, contracted and implemented
• arrangements for the return of required data to corporate environments are not identified, contracted and implemented
• inappropriate data is subject to standard purge cycles deployed by service provider.

Risk: Devolution of retention and disposal decisions to individual teams and users

Today individual teams and users are able to acquire systems and define management rules around these without any reference to corporate strategic or governance policies, including information retention and disposal rules.

Often these teams and users are unaware that corporate information retention and disposal rules apply to the work they are doing, and therefore fail to consider them when making decisions about what information should be kept at the end of a project.

Once these individual destruction decisions are made in these systems, there is little possible recourse. When the information is gone, it is gone. For example, the National Records and Archives Administration in the USA advises that:

After a Google Apps user or Google Apps administrator deletes a message, account, user, or domain. and confirms deletion of that item (e.g . empties the Trash) the data in question is removed and no longer accessible from that user’s Google Apps interface.

The data is then deleted from Google’s active servers and replication servers. Pointers to the data on Google’s active and replication servers are removed. Dereferenced data will be overwritten with other customer data over time.

Where to find the rules governing the retention and disposal of government business information

These rules are contained in documents issued under the State Records Act 1998, called retention and disposal authorities. These documents reflect the business needs for various types of information, as well as the wider accountability requirements for information.

If you need help to locate the retention and disposal authorities that apply to your area of business, please contact State Records, or the records and information management staff in your organisation.

Recommended strategies to effectively deploy the rules governing the retention and disposal of government business information

We recommend that you:

• foreground information retention and disposal requirements in all system and service decisions
• use retention and disposal authorities as active planning and management tools, to use in system and service decisions
• deploy strong organisational information governance frameworks that assess business needs and risks globally across the organisation and deploy targeted and strategic information management solutions in response
• build collaboration between business, ICT and records and information management staff
• raise awareness among project or business staff of the need to consider information retention and disposal in the use of all collaborative tools
• prioritise actions, focus on high risk/high value systems and medium to long term information retention requirements (where information is required for 5+ years)
• specifically identify and assess the systems containing information that needs to be retained 10+ years and determine continuity strategies to ensure this information can be accessed, trusted and used for as long as it is legally required to be kept.

Has anything else worked for you? Please do let us know!