Future Proof – Protecting our digital future

We revisited digital disposal at our recent Digital Implementers Group meeting. We were interested in how things had progressed since the topic was last discussed. Were the main drivers for digital disposal still compelling? Were the main blockers still impassable? After an energetic discussion, we knew we faced familiar challenges, but also included are some thoughts on potential ways forward.

Driving digital disposal

The two main drivers that emerged in the course of our discussion were the costs associated with digital storage and the difficulty of servicing Government Information (Public Access) Act (GIPAA) requests.

While acknowledging the falling cost per unit price of digital storage, we noted costs related to backing up increasingly large data stores, and the ever-present drive to find savings through efficiencies. At this stage, however, we were only able to consider these costs in an abstract sense. To better inform a business case, we would want to be able to differentiate between records that need to be retained and records that are due for immediate destruction, and the storage costs for each. This would show us how much money we would stand to save.

Servicing a GIPAA request can be difficult if you don’t have good oversight of your agency’s records holdings. None of our members reported any enterprise search tools that covered their full array of systems, and the challenge is compounded where legacy and decommissioned systems exist in a state of purgatory. Ostensibly, removing redundant, obsolete and trivial records would make the remainder easier and faster to search.

Main blockers

We know that we can save storage costs and time spent backing up data if we destroy unneeded records. Why haven’t we done this? The group tended to agree that the business case had not been compelling enough to convince senior management to invest in digital disposal – though we think it is moving towards a tipping point. As noted above, not knowing what proportion of your holdings is eligible for immediate destruction means we can’t quantify the potential savings to storage costs. The cost of sentencing your entire digital holdings is likely to outstrip the costs for storage for many, many years.

The group also discussed how non-records colleagues perceived digital records. Not all people think of records in business systems as being ‘records’. In an example related by a group member, we were told of someone that had suggested that the physical component of hybrid files be destroyed, and that the digital component would just be kept, simply because it was digital! Convincing people that digital records can and should be destroyed appropriately is an important step to convincing them that the activity of digital disposal is worthwhile.

Another issue that was raised was the question of how to separate out records with different retention periods within a system. This is particularly relevant if the system contains a mix of temporary records and State archives.

Challenges

Based on the discussion, it seems that we are struggling to get far from the starting block. At the risk of sounding dramatic, we are still fighting a battle for hearts and minds rather than the battle against bits and bytes. So even after we win the people over, it’s good to know we have our challenges lined up in deciding how to approach the technical task of digital disposal.

We need to have control of our digital records. We need to identify where and why they are being created, what they are/can be used for, and how long we need to keep them – all of which go to the risk and value of the records. Good information governance is the key. We also want to minimise the future accumulation of unneeded digital records. To do this, we need to assess upstream business processes and find ways to make the records as disposal-ready as possible.

We’re firm believers that we need to give most attention to our high-risk, high-value records. In order to do this, we want to find ways to dispose of our low-risk, low-value records quickly and with minimal intervention. However, we need to do this accountably and be able to demonstrate that the level of treatment is commensurate with the importance of the records. To manage this, we need to have a policy approach that helps us to determine our risk and value thresholds, and a technical or mechanical approach to enable the actual sentencing.

Discrete and non-discrete records

This is another way of categorising records. A discrete record is typically an unstructured, text-based record like an email, Word document or PDF. It is discrete because conceptually, it is a single, stand-alone object (or bundle of objects) that we can append metadata to.

A non-discrete record is not a neat, stand-alone object. It is comprised of dynamic elements, and the information that is presented to the reader can vary depending on what or when it is requested. The typical example is a database.

We make this distinction to see if we need different strategies to dispose of these records. For most part, we have tried to manage discrete digital records in the same way we manage paper records (which are also discrete records). We put them in folders and files and, as best we can, persistently link them to their relevant recordkeeping metadata.

In theory, we dispose of them by recording sentencing decisions against each individual record in the control record. Then, after the appropriate checks and balances and at the appropriate time, we execute the appropriate disposal action. This is easier when the records have been well controlled, meaning good metadata such as titling and functional classification. It is much harder when these records management practices have been neglected.

In most respects, disposing of discrete digital records is a known process, because it is so similar to the paper process. However the issues created by volume are exacerbated many times over. At some point (that has probably already passed), it will no longer be an issue we can resolve by chaining more monkeys to more typewriters – we may need to take a qualitatively different approach.

For non-discrete records, there is no paper analogy to borrow from, so we know we need to take a new approach from the start. There are a number of issues we need to address. In many instances, there may not be any obvious control record to append metadata to. A database may also be unable to support permanent spot removal of data without affecting its overall stability. And, importantly, how do we decide what constitutes the record that we need to sentence and destroy?

We’re discovering these issues as we continue to tackle digital disposal. We’ll take a closer look in a future blogpost.

Pathways to the future state

As we move towards that tipping point where we can all agree that keeping all digital records and information is no longer viable, we want to have all our ducks in a row. This means having defensible disposal policies in place, including a policy on normal administrative practice (NAP) in your organisation. A clearly articulated NAP policy, effectively implemented, could help you to prevent the accumulation of entire categories of digital debris. For example, if you have an automated and well-documented procedure for capturing business emails in your corporate recordkeeping system, then it might be an appropriate normal administrative practice to routinely purge all emails from inboxes that are more than four weeks’ old.

We also need to have a credible plan to manage the sentencing and disposal process, whether we are dealing with discrete or non-discrete digital records, and whether they are in high or low volumes.

If you have had success with any of these things, or if you’ve been down a path that wasn’t so successful, we’d like to hear from you.

Image credit: ‘Digital Analog Waste 1’ by Steve Janosik is licensed under CC BY-NC-ND 2.0.