Future Proof – Protecting our digital future

As usual, Government Recordkeeping staff have been asked lots of interesting questions about digital information management and recordkeeping issues in March and April.

Here is a sample of some of these questions and our responses.

• Does converting emails to PDF before I save them into our records system make a more authentic, accountable and long term record?
• We are implementing a business system to support one of our major, high risk business areas. What types of recordkeeping requirements does the system need to support?
• What do I need to do with my legacy data once it has been migrated? Can I just delete it?
• We have been scanning our incoming paper correspondence for many years and day boxing it, but we have never destroyed the paper records. What should we do?
• Is there a standard practice for archiving Facebook posts?
• Are drafts official records?
• Is it a better use of money is paper records are scanned and saved as digital records, rather than funding more storage for paper records?
• With our scanning project, what kinds of quality control benchmarks should we implement?

Does converting emails to PDF before I save them into our records system make a more authentic, accountable and long term record?

Although PDF is a good, stable, long term format, we do not recommend that you convert emails to PDF before saving them.

Converting email messages to PDF is akin to printing messages and filing them on a paper file. This approach is better than not capturing the messages at all, but is not an ideal recordkeeping solution.

The recommended method for capturing email messages as records is to capture them directly from an email application (for example Groupwise or Microsoft Outlook) to an electronic document and records management system (EDRMS). Depending on the EDRMS and the way in which the integration with the email application is configured, this approach is likely to capture all of the metadata necessary for understanding the context and content of the message over time so that it can be relied on as evidence of the business that occurred.

This automatic and thorough metadata capture may not be possible with PDF conversion.

A recent report on email preservation (Preserving email, DPC Technology Watch Report 11-01 December 2011, p. 26) notes:

“When [Adobe] Acrobat Professional has been installed on a local workstation that also has Microsoft Outlook, a menu item will be added to Outlook, allowing users to save individual messages or groups of messages to a PDF file or PDF portfolio. However, messages saved in these formats will experience a loss of fidelity. In particular, portions of the header will be excluded, and attachments will be encoded (in an unspecified format) inside the PDF file. Such a roundabout method of preservation poses risks.”

This report also notes that a 2009 research project in the UK found that 14 properties of a message header and 50 properties of a message body contributed information that established the authenticity and integrity of an email message. Again, some of these properties may be lost or only partially represented in a PDF version of the email message.

So our advice is capture your emails into your EDRMS in their native format.

We are implementing a business system to support one of our major high risk business areas. What types of recordkeeping requirements does the system need to support?

What needs to be supported here are your business requirements. Irrespective of what technology solution you implement, you need to make sure that the business will have the information it needs, now and into the future.

You basically have two different options available to you.

Firstly, you can make sure your business system is able to perform as a recordkeeping system. This means the system would be compliant with State Records’ Standard on digital recordkeeping and comply with core requirements in the Standard such as:

• the system can capture read only versions of the digital records you require
• the system can retrieve and present the defined digital records in human readable form
• the system can restrict or permit access to defined records if required
• the system can make and keep metadata that allows information to be used, managed and understood
• the system can export records and their metadata when required.

Your second option, if the business system itself cannot meet the requirements of the Standard is to work out a program whereby you regularly export the necessary business records out of the business system and capture them into your corporate EDRMS as a way of ensuring you have the records needed to support ongoing business operations and associated legal or accountability requirements.

What do I need to do with my legacy data after it has been migrated? Can I just delete it?

You actually need to keep legacy data for a short period post migration.

State Records has a disposal authority with the catchy name GA33, Source records that have been migrated.

This authority says that you can destroy your legacy or source records provided that:

• your migration was appropriately planned, documented and managed
• your pre and post migration testing shows that your migration was successful
• you have kept your source records for a period of no less than six months following the successful migration of the records (depending on the business and risk value of your legacy data, you may decide to keep the legacy records for longer than six months). This six month validation period allows appropriate time to verify and re-perform any migration activities should this be required.

In GA33, the section Conditions for the destruction of records has a full description of the factors you should consider before destroying your legacy data.

We have been scanning our incoming paper correspondence for many years and day boxing it, but have never destroyed the paper records. What should we do?

There are no problems with destroying your paper records post their digitisation.

In State Records guidelines on digitisation, one section covers the destruction of original paper records.

Part C of this section discusses the authorisation required for this destruction. It outlines how you should check off all necessary considerations before destroying the paper original records. In relation to the requirement to ensure that ‘All requirements for retaining originals have been assessed and fulfilled’, it says that you should ‘Obtain and document approval from a senior manager if destruction is to take place’.

This means that you should seek written approval from a senior management for your proposed destruction. This approval could be for all day-boxed records in the period you are looking to destroy – there is no need for detailed document level approval.

Before destroying the paper originals you just need to make sure that there is no internal business need for any of the paper originals, and be sure that the scanned versions of all these records are clear, accessible and well managed.

The Frequently asked questions section of the guidelines might also provide some extra information on this issue.

Is there a standard practice for archiving Facebook posts?

There is not a standard practice for archiving Facebook posts.

Some Facebook posts will be minor and administrative and will not need to be kept. Others however will have ongoing business value, will capture community and strategic input, will demonstrate community consultation or engagement or will form an important ongoing record of business operations.

For these types of posts, each organisation needs to determine the recordkeeping strategy that best suits their business purposes.

Some organisations may still have a fairly rigid Facebook post approval process and so records of their posts may be captured in official approval documentation.

Others may capture screenshots each day, week or month and save these as PDFs into their EDRMS.

Others may use emerging recordkeeping tools that operate with Facebook and a range of other social media channels.

Others may use some of the cloud-based third party offerings that are able to provide regular downloads of social media content.

Others may use social media monitoring tools to capture information about organisational posts and community responses to them. There are lots of options, many of which are referenced the State Records guidance, How do I capture social media records? which looks at the range of different systems people are using at present to make and keep social media information.

Irrespective of which tool or procedure you follow, you do need to make sure you have a basic core of information that gives your record some evidential and accountability value. You need as a minimum the name of the user account, the text of your post and the date and time of posting. Depending on your business needs you may have additional requirements, such as a need to know details of links posted (full details, not just a shortened URL), a need to capture community responses to your posting, metrics about its reach or impact, etc.

Much more information about social media recordkeeping is available in State Records draft guidelines, Strategies for managing social media information: How to keep records of government social media business.

Are drafts official records?

An ‘official record’ is a record made or received by an agency in the conduct of its business. According to this definition, drafts are records.

However, drafts do not always need to be kept. State Records advises that drafts should be kept when there is an identified recordkeeping requirement to retain them because they document significant decisions, reasons and actions or contain significant information that is not contained in the final form of the document, or both. Examples of such drafts include:

• drafts containing significant or substantial changes or annotations
• drafts relating to the formulation of policy and procedures, where the draft provides evidence of the processes involved or contains significantly more information than the final version of the record.

In determining whether drafts should be captured in your corporate records systems, you might also like to think about the business they document and their retention period – not all drafts are created equal!

For example, draft versions of policies and strategies for the management or administration of the public sector workforce that contain significant changes/alterations or were formally circulated for comments are required as State archives under FA329, but draft versions of an organisation’s internal records management policies have a minimum retention period of 3 years after action completed under GA28.

State Records has published two leaflets for public offices that summarise employees’ recordkeeping responsibilities and prompt the creation and capture of records. These leaflets could be a useful support to your procedural guideline. Downloadable leaflets What have records got to do with me? and Recordkeeping reminders are available to remind staff about their recordkeeping responsibilities and the Recordkeeping reminders leaflet has a short section on the types of drafts that people should capture and keep as records. NSW public offices can also order hardcopies of these leaflets (the first 100 of each are free) by contacting State Records.

Is it a better use of money if paper records are scanned and saved as digital records, rather than funding more storage for paper records?

Firstly, it is important for all organisations to manage the quantities of records they hold by undertaking regular disposal of time-expired records in accordance with the authorised retention and disposal authorities that are relevant in your organisation.

Once you have done this, a program to digitise records can reduce paper records storage costs, however there are considerable costs involved in establishing a digitisation program.

For this reason, it’s important to develop a business case to ensure that all the requirements and costs have been accounted for prior to the commencement of the program, particularly so that a clear understanding of the return on investment can be determined.

Decisions also need to be made about what will be digitised – digitising mail, invoices, applications, submissions etc. as they are received by the organisation, and/or digitising existing records to enable the destruction of original paper records, make records more accessible or to preserve old and fragile records.

The business case process is also important as assessments of the maturity of the organisation’s recordkeeping and the organisation’s electronic document and records management system (EDRMS) need to be made.  The digitised records will need to be stored in the EDRMS and the EDRMS must be able to support the capture and maintenance of the digital record over time.

State Records has detailed digitisation guidelines available that provide extensive advice about assessing the viability of and then implementing comprehensive digitisation programs.

With our scanning project, what kind of quality control benchmarks should we implement?

State Records guidance on digitisation projects and programs is separated into two parts, one focussing on business process digitisation (i.e. ongoing routine digitisation of records for daily business use) and the other on back capture digitisation (i.e. retrospective or project-based digitisation of existing paper records).

The sections on business process and back capture digitisation both include advice on quality checking samples of images, including the following suggestions:

If sampling is adopted, benchmarks for the frequency of sampling should be determined according to system usage and expected or anticipated deterioration periods. Sampling should include assessments of both digital images and their metadata. Advice from system vendors may assist in determining the frequency period.  The extent and frequency of sampling should be documented.

Initially, it may be appropriate to sample frequently. However, once benchmarks, equipment and processes have been stabilised, this may be reduced to a random sampling of between 5 and 10%.

Care must be taken to ensure that samples are representative of the range of records digitised and include examples where the quality of the original paper records is poor compared to the majority of sampled original records. In some cases, e.g. following equipment repairs or if using new staff or service providers, each image may be checked until there is confidence that the standards are being met.