Future Proof – Protecting our digital future

At our most recent EDRMS Implementer’s Group meeting, we discussed digital disposal.

Specifically we discussed:

• are organisations using disposal authorities to sentence, retain or destroy digital information?
• are there drivers for or barriers to digital disposal?
• what are some main issues that might be impeding the implementation of digital disposal?

Lots and lots of interesting issues were discussed:

Digital disposal results from State Records 2013 Digital recordkeeping survey report

We started with an overview of what State Records’ whole of government 2013 survey on digital recordkeeping said about disposal.

The State Records 2013 Digital recordkeeping survey report showed that, in terms of EDRMS:

• 11% of public offices had disposed of digital records in EDRMS in the last 6 months
•  4% had disposed in the last year
• 8% had disposed in the last 2 years, and
• 77% have not disposed

In terms of authorised disposal occurring in other systems used by public offices, the survey respondents said that:

• 38% will address disposal of digital records in business systems in the future
• 30% don’t dispose of any digital records in business systems
• 21% are likely to keep all digital records in business systems
• 19% will dispose as part of system migrations
• 17% will dispose as part of system decommissioning, and
• 5% don’t know how the organisation disposes of digital records in business systems.

Survey participants also gave a variety of valid reasons to explain why they were not doing more about digital disposal, including:

• records in the corporate EDRMS have not yet reached the end of their retention period
• the EDRMS has only been implemented in the last 7 years
• the organisation has not implemented retention and disposal authorities into the system due to system issues
• no retention and disposal authority exists for the organisation
• decision has been made by organisation to retain all records, particularly as there is ‘no imperative to dispose of digital records as there is adequate storage capacity’, and
• there are limited resources to implement digital disposal.

Some of the risks that lack of disposal can bring

State Records’ concern however is that when digital disposal is not done ‘by design’ and when these issues are not foregrounded and considered proactively, the following can very easily occur:

• increased data storage costs as more organisations store more and more data in systems
• system inefficiencies as there is no disposal of data and systems have to process more and more data
• increased migration costs as more data is migrated forward
• loss of high value data in amongst all the ‘noise’
• increased risk of inadvertent data loss through large-scale data purging
• increase of poorly controlled and managed data retained sitting in legacy systems, and
• more data means slower retrieval or ineffective retrieval of the data required by users.

 What was said at the EDRMS implementers group about digital disposal

As usual, great discussion abounded at the meeting and the consensus was that digital disposal is not occurring because essentially it is a wicked (ie very complex) problem.

There are multiple reasons behind why it is not occurring and here are some of the issues we discussed at the meeting.

 EDRMS technology can actually make it hard

Disposal is often not easy or intuitive in these systems. Often, disposal functionality is also an add on and is not always rolled out at the start of an EDRMS implementation. And as the implementers said, disposal is not where the immediate business needs and drivers for EDRMS projects are: ‘we concentrate on getting the stuff in, not getting it out’.

Data quality is often not good and this makes it hard to trust the information in the system and trust that you can actually get rid of it

A large number of participants reported on this trust issue. Users are populating EDRMS incredibly quickly with digital content and this is great, but there is often very little oversight or checking of the quality or consistency of user filing techniques and their accuracy. Some participants reported that their rough assessments are showing that user accuracy may not be terribly good and therefore they are fearful of doing disposal because they are worried about inadvertently destroying information that has been placed in the wrong location.

Good digital disposal will come down to good governance and good change management

At present, business and the information that documents it is transitioning very quickly to digital environments. This can result in information environments that could be described as more chaotic than well managed. The consensus of the group was that once:

• good structures are in place
• users understand governance frameworks and how business is expected to operate in new digital spaces, and
• good trust around systems exist and good rules have been established

these governance and change management strategies will create the kind of trusted environments where disposal can much more easily occur.

Numerous attendees reported that issues around document profusion are already impacting their business environments, such as uncertainty about which is the final or official version of a document, with the result that everything is kept and nothing gets destroyed. So good governance and good change management seem to be core to establishing good and trustworthy disposal practices into the future.

 Records people are not yet sufficiently involved in the conversations that will lead to ‘by design’ solutions

Within the 20+ sized group, only 2 of the records managers present had ever been asked by business or ICT staff to advise on the retention requirements that apply to the information to be contained in new or evolving systems.

Both of these records managers however reported that they had found their disposal authorities easy to use in this context. The systems they were asked to advise on related to a specific area of business and so their disposal authorities provided very specific requirements that could map quite directly to the proposed system functionality.

In both cases the records managers also reported that the relevant business and ICT staff were very surprised to hear that there were actually official rules around retention that they needed to consider, but in both instances business and ICT staff were grateful to have official guidance and rules to base their system design requirements on.

So again, promoting good governance and awareness of the rules that do in fact apply to all forms of government information, and promoting these to the people who are charged with making decisions about this information in evolving business frameworks, can lead to significant changes and improved disposal outcomes.

Because of the volume of day to day work, disposal is still seen as a specific project that will be deployed at some stage in the future

For most records managers, however, the day to day work is itself overwhelming, leaving little time to plan and implement strategic and proactive digital disposal frameworks.

In most organisations there is still a project based understanding of disposal and it is seen as a one off event, deployed as a major project once every few years. Using disposal authorities as proactive planning or communication tools is not widely undertaken in most organisations.

Disposal is not being considered in cloud transitions

In terms of the records managers in the group, no one has been asked to provide advice on cloud transitions in their organisations, to let relevant business and ICT staff know how long information that is transitioning to the cloud needs to be kept for.

The group discussed how awareness of retention requirements is actually really important in the cloud, particularly at contract design and in early scoping and budgeting discussions, where future data portability requirements need to be adequately scoped and understood.

Big bucket solutions are very appealing and are being deployed in different organisations

A number of public offices are working on deploying ‘big bucket’ versions of their disposal requirements.

One records manager reported that she had good corporate support from consultants and other business staff for her ‘big bucket’ approaches that have rolled up all the organisation’s disposal rules into three main categories. This approach is really facilitating the design of their planned SharePoint implementation and has helped to build corporate support and understanding for the work of the records team.

We can’t fix everything so we need to focus on what is most important

At the meeting there was good consensus around the need to base strategies for digital disposal on business risk and value. We can’t fix everything so we need to focus our strategies on the systems, information and records that are of most value to our organisations.

Every day, problems with data over retention also compound, so it is very important to initiate action now, start small and learn and improve as we go, rather than waiting for perfect, organisation-wide solutions.

And none of these issues is actually anything new

At the conclusion of this discussion it is worth emphasising that the problems with digital disposal are very similar to those affecting paper record disposal. Legacy volumes of unsentenced paper records are a significant issue in most organisations today and the issues that have caused this are very similar ‘wicked’ problems to those described above. So we need to learn the lessons from this experience and prevent legacy digital data volumes from accumulating and creating irreconcilable problems into the future.

And finally, a big thank you as usual to the members of the EDRMS Implementers Group for your enthusiasm and great discussions and ongoing commitment to sharing your knowledge and expertise. See you at the next meeting!