Future Proof – Protecting our digital future

Many organisations across the NSW public sector have well established digital recordkeeping systems. As these systems have been in place for some years, they contain records that are able to be destroyed in accordance with approved retention and disposal authorities.

A number of organisations have recently contacted us to discuss the requirements for keeping metadata for records destroyed in such systems:

• What metadata needs to be kept?
• What level does this metadata need to be kept at? (i.e. file level, document level etc.)
• How long does this metadata need to be kept for?

As is often the case, the answers to these questions are likely to depend on the individual organisations’ business and the particular risks relevant to that business. In short, ‘it depends’!

What metadata do organisations need to keep for records that have been destroyed?

The Standard on Records Management requires that NSW public offices manage their records and information well. Public offices can comply with this requirement by systematically and accountably destroying records and information when it is legally appropriate to do so.

In order to account for records and information that have been destroyed, public offices need to keep certain metadata.

Metadata for files vs metadata for documents

Many organisations apply disposal decisions at the file (or folder) level. Retention periods and disposal actions are then inherited by all of the documents in the file.

This approach mirrors the way in which disposal decisions are generally applied to paper records.

In the paper world, organisations usually did not maintain document level indexes with information about each document placed on a file. When they destroyed a paper file, they therefore could only keep metadata about the file as a whole, not about each document in the file.

However in the digital world, many systems create such indexes by default. This gives organisations the opportunity to retain document-level metadata if it would have ongoing value to an organisation in terms of being able to account for disposal actions.

Understanding the risks

If organisations were to routinely retain all document-level metadata they could end up in the situation of keeping as much (or even more!) data as they were destroying.

Organisations should instead assess the risks associated with their business and determine if there are any types of records for which it would be appropriate to keep more than the file-level control metadata.

For certain high risk processes, knowing what documents were on the file that was destroyed could be critical. In these cases it may be appropriate to retain some document-level metadata such as document/record number, title and date created.

For other processes, it may be enough to be able to say that a particular document was on, or was likely to be on, a particular file, and that file has been destroyed.

Knowing the metadata

Organisations should also assess the quality of the metadata that has been captured at the document level. Its ongoing value will depend on what information was provided (either by users or automatically generated) when the document was captured.

If the quality of the metadata is poor, there may be little point in retaining it after the documents to which it relates have been destroyed. If this is the case, organisations can seize the opportunity to improve the quality of document-level metadata for future capture – see Metadata for managing records and information for further advice.

How long does metadata need to be kept after the records to which it relates are destroyed?

The general retention and disposal authorities for administrative records (GA28) and local government records (GA39) require certain metadata to be kept for 20 years after action completed:

• ‘primary control records’, including metadata, for records that are not required as State archives (GA28, 12.9.3; GA39, 16.8.10)
• records relating to the implementation of disposal decisions (GA28, 12.11.1; GA39, 16.8.5).

Retention and disposal authorities relevant to individual organisations may also outline specific retention requirements that apply to certain forms of metadata.

These retention periods reflect that organisations must be accountable for the destruction of records and information. Organisations must be able to say with certainty that a file (or certain documents associated with it) was legally destroyed on a particular date in accordance with a particular authorisation.

As always, the retention periods specified in retention and disposal authorities are minimum periods only. Metadata for particular records and information may be required for longer periods, depending on the risks associated with the conduct of the business and the potential for or likelihood of the organisation being required to provide evidence of the destruction. These should be determined in relation to the organisation’s business needs.

As always, we would be very interested in hearing from public offices that have destroyed digital records to find out what decisions you made regarding the retention of metadata – what metadata did you keep and why? And how did you make these decisions?

Photo credit: Nottsexminer – “Tawny Owl” (CC BY-SA 2.0)