Future Proof – Protecting our digital future

Sorry it has been so long since our last Q&A post but we have honestly been super busy answering enquiries!

People within the NSW public sector have been asking questions about lots of fascinating issues, relating to changing business frameworks and how recordkeeping should be used as a support to these.

Here are some of the main issues we have been providing advice on. Hopefully some of this advice can help you too:

• Is GIS data a record and if so, what are strategies for managing it?
• When might we need to consider managing data as a record?
• Staff are scanning invoices direct to our Finance system and storing and managing them there. Is this OK?
• Do I need to keep records of calendars and task lists from Outlook?
• If we are just using social media to generate informal debate, not as a formal consultation tool, and if more formal submissions are captured elsewhere, do we have to capture social media records and keep them for long periods of time?
• People in our organisation are scanning to the hard drive of our multi-function devices (MFDs – photocopiers, scanners etc). Is this a problem?
• Concerns over some devices for document scanning

Please remember, we always welcome discussion on and comments about the advice we provide. If you have any feedback or other opinions, please do leave us a comment below. Digital recordkeeping practice is evolving all the time so please do help us continue to refine and improve our strategies and advice with your feedback.

Is GIS data a record and if so, what are strategies for managing it?

Lots of government organisations use real-time geographic information system (GIS) data as the basis for decision making. For example, in local government, decisions around billing, planning, waste management and other activities are actually based on geographic data.  In this example, to enable councils to justify and explain these billing and planning decisions, it may be important for a view of the GIS data to be maintained as an authentic and accountable record at the specific point in time when a decision is made.

If a point in time representation of the data is not captured, the frequent changes and updates to GIS data can mean that it will not be possible to reference back to the actual dataset used as the basis for financial and other decisions.

GIS themselves are dynamic systems and do not generally have the capacity to store fixed representations of data. Therefore, to keep a record of the data as it stood at a specific point in time, a suggested practice is to perform an export of the GIS database including its metadata, users and permissions as XML files when required, and to capture these into a corporate recordkeeping system.

This level of recordkeeping does not need to be a standard process whenever a business decision is made using GIS data as a reference. However it should definitely be considered to support and account for major actions, decisions or transactions that are based on GIS data.

When might we need to consider managing data as a record?

We also responded to this closely related question.

Data best meets business needs when is kept and managed as active business data. Large and comprehensive records of your dynamic business data generally do not need to be made.

However, all aspects of digital business practice are undergoing radical and rapid transformation and we do need to ensure that business information needs and risks are fully considered in this transformation.

To support this, you could work with your data managers to identify whether there any scenarios where point in time views of data are necessary and should be captured as a record. Such scenarios could include fire or emergency situations or points in time when major business decisions are made.

In these types of high risk, high consequence scenarios, it may be important to have a fixed view of the data as it stood at that specific point in time. Many dynamic database environments cannot maintain this point in time view of data, or can only provide it for a short period of time.

If specific points in time are identified where business requirements mean that data should be captured as a record, plans should be developed to support this. These plans will generally involve exporting the identified data and any metadata needed to interpret it out of its native database environment and into a stable storage environment. If this process involves transforming the data into a standard format, like PDF, you should ensure that this transformation does not involve the loss of significant information.

For long term accountabilities, organisations should not rely on database roll back functionality because this might only last for 3-5 years or whatever the life of the system is. Business or legal accountabilities for high risk activities will likely last for much longer periods.

Another consideration for organisations is the lack of data outputs that exist generally today. Previously for instance, a lot of geographic data was output on a regular basis as a map. Today, most people interact with live data online and much of it is never now output in a report or in a document such as a map. Businesses should consider if, in 5 or 10 years, is this going to impact on their business operations or capacities in any way? For example, is your organisation ever likely to need a map to show what its geographic boundaries looked like in 2013? Could it be worth building in annual captures of certain datasets in an accessible output form to serve future business needs?

And this brings us to our final point, retention. Some data has very long retention requirements. The complex, proprietary technologies in place today to capture, overlay, interpret and present data potentially cannot be maintained through time. Plans are generally not in place for the long term management and archiving of core datasets but they need to be. For very high risk, high value datasets, you and your corporate data managers should consider regular export strategies to ensure records of these key datasets can be maintained.

Staff are scanning invoices direct to our Finance system and storing and managing them there. Is this OK?

This situation is increasingly common. Some agencies like Police are integrating their invoicing and payment systems with their EDRMS. This integration enables to them manage payment data in their Finance system and manage scanned invoices and other documentation in their EDRMS. These integrations however are in the minority. Many organisations today are scanning  invoices direct to their Finance systems.

This is a valid approach. Most finance systems are well designed with strong audit, reporting and accountability capacities. Information within these systems also does not need to be kept for extensive periods of time (in general, the bulk of financial information only has a 7 year lifespan) and finance systems tend to be fairly stable, with the capacity to maintain information for as long as it is required for legal and audit purposes.

It is worth, however, having a conversation with your Finance staff and help them to fully assess their system and their approach to scanning, to let them determine whether this approach will best meet their business needs.

For example, in the Police case study referenced above, in explaining their EDRMS/SAP integration Police said:

Another important aim was to make sure that the records within SAP would not create an ongoing storage and legacy problem for the organisation. Large financial processing systems like SAP are good at making data but they are unable to manage data efficiently over time. The Police recognised that by integrating SAP with their EDRMS they could ensure that the invoices could be managed as records and could be sentenced with a disposal authority and trigger. This would then streamline disposal when the 7 year retention period expired, bringing additional efficiencies for the organisation.

It would be worth having a conversation with Finance staff to ask if their Finance system can routinely and accountably purge invoices at the end of their retention period. If it has the functionality to do this, then that is excellent. If it doesn’t, then they should assess the potential implications of this.

For example, if a large number of invoices are to be scanned each month, it could be worth assessing how many would then be stored in the system at the end of a 10 year period. What would this amount of legacy data storage mean to system run times? If keeping hold of these scanned images in the system for 7+ years is really going to impact on system functionality and operations, then possibly integration with an EDRMS that can routinely and accountably destroy these records when their retention periods expire is a viable option.

Given these invoices will likely have to be kept for 7 years, it would also be valid to ask the Finance team whether the finance system can actually migrate invoices together with their relevant transactional metadata into a new system if the finance system changes or is upgraded. If it is likely that maintaining these connections will be challenging, or if it means that a lot of legacy and non legacy data would have to be uniformly purged and thereby creating potential accountability or audit problems, then this too might be a driver for EDRMS integration, in order to prevent these potential problems.

As the Standard on digital recordkeeping makes clear, integration with an EDRMS needs to be a business and risk-based decision, not a compliance-based decision. Finance systems generally have strong audit and accountability structures and these might provide all the functionality your Finance team needs to manage invoices appropriately and to consolidate access to all related financial information. It is however the routine disposal and maintenance of scanned invoices for 7 years that might prove more challenging for the finance system. Your Finance team should determine whether these management issues could cause genuine data management problems in the medium to longer term and if so, whether integration with an EDRMS could be a good solution.

Do I need to keep records of calendars and task lists from Outlook?

For some staff accounts, yes, you probably should. For certain staff, calendars or diaries can be long term, important records that you will want to keep.

Screenshotting is an option but Outlook does have the capacity for printing or emailing different calendar views. To make and keep records of their calendars, identified staff could email a monthly export from their calendars which could then be captured as a record in a corporate system. Summary views of email calendars can also be printed and scanned as a record.

For identified staff or positions you could develop a procedure to require the regular export and capture of designated calendars at defined intervals. Some configurations of Outlook implement a fairly short delete cycle on calendars (sometimes approximately every 6 months), so it would be important to ensure a regular calendar capture process if your Outlook system is configured in this way.

For Task lists, from within Outlook a task can be forwarded in an email with a small attachment outlining the specific details about the task, its due dates, priorities etc. This forwarding needs to be done on a task by task basis, so it should be determined whether this level of recordkeeping for tasks is required in your organisation.

The need for task management of this type would need to be determined by the nature of the tasks being performed. If there are long term accountabilities or if your organisation has to regularly report on what steps were done when, then possibly there is justification for detailed task capture. But if the task option in Outlook is generally used by individuals to manage their own times and deadlines and the main business information and accountabilities are captured in the outputs of these specific tasks, then possibly there is less justification for detailed task management.

If we are just using social media to generate informal debate, not as a formal consultation tool, and if more formal submissions are captured elsewhere, do we have to capture social media records and keep them for long periods of time? (We have really long retention requirements that apply to our records of formal public consultation)

With your informal social media debate, yes, your recordkeeping decisions should be based on your actual business use of social media. If it is very informal, just used to generate ideas but then people are clearly directed to formal consultation processes and if these processes then have strong recordkeeping to support and maintain them, then you may decide that you don’t need to formally capture records of these specific social media conversations.

If your business decisions and operations are supported solely through the traditional formal consultation and it is clear to the public using social media that this is the case, then you don’t need to keep everything, just the right information that you will need to support future decisions, reporting, reuse, accountabilities etc.

However, social media is changing, government use of social media is changing and public expectations for social media are changing too. You do need to ensure that your recordkeeping strategies keep pace with this change if necessary. For example, it could be that more people start contributing ideas through social media rather than formal submission processes. People might start asking more questions, reporting on issues and engaging with staff via social media. The organisation might also increasingly have actions and decisions that are based on information communicated through social media, or the public might increasingly act on information your organisation is disseminating via social media.

If any of these changes are taking place, you should definitely consider strong recordkeeping frameworks for your social media accounts where these types of discussions are occurring. Advice on how to keep social media records is contained on the Future Proof blog.

People in our organisation are scanning to the hard drive of our multi-function devices (MFDs – photocopiers, scanners etc). Is this a problem?

Many multi function devices (MFDs) like photocopiers and scanners are configured to enable scanning to their hard drives. If documents are scanned to these hard drives, these drives become storage environments for this data. MFDs however are not appropriate storage environments for corporate records. They are generally uncontrolled, insecure storage environments.

MFDs in government are also generally managed under contract with vendors and are subject to regular upgrade and replacement. Under this regular upgrade and replacement process, an MFD containing a large repository of your scanned data could be removed from your organisation and potentially accessed by others because comprehensive data sanitation has not occurred.

This is not a speculative concern. As discussed on the Privacy and Information Security Law Blog, by August this year, the Office for Civil Rights in the Department of Health and Human Services in the United States had so far imposed $3.6 million dollars in civil penalties on organisations that had failed to erase photocopier hard drives before returning them to leasing companies.

State Records advice, Destruction of records, says:

For public offices to maintain appropriate control over the destruction of digital records stored on digital media, some media may need to be sanitised at appropriate times and by appropriate methods. There are many horror stories of information abuse and illicit information collection through the obtainment of hardware that has not been appropriately ‘cleaned’ or sanitised.

Any record stored on digital media is particularly vulnerable to abuse and illicit collection. Appropriate methods need to be taken to ensure that when a record that is stored on digital media is ready to be legally disposed of, it is safeguarded against potential misuse.

Not all media can be sanitised. Some media must be destroyed. Media that is suitable for sanitisation includes some magnetic media, erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), volatile memory and non-volatile memory devices such as USB removable media, pen drives, thumb drives, flash drives and memory sticks. Other examples of media that can be sanitised include electrostatic memory devices within printers and photocopiers…

It is very important to ensure that service contracts, procedures and other measures controlling the use of MFDs in your organisation control and manage the business information that may be stored on your MFDs.

Ideally no important, personal or long term value business information should be stored on an MFD. Where it is, however, strategies must be in place for wiping this data from the device before it is recycled, thrown away or returned to a leasing agent and ensuring that this data is instead maintained in secure corporate systems if it is needed for ongoing business, client, reporting or accountability purposes.

Concerns over some devices for document scanning

Finally, thanks very much to our excellent colleagues at the Public Record Office in Victoria (PROV) who very quickly noticed this issue and promoted it in their blog post, Concerns over some Xerox devices for document scanning.

PROV reported on the research of computer scientist Daniel Kriesel who noted randomly altered numbers in some scanned documents. The good news is that Xerox has subsequently issued a patch to resolve the character substitution issue that was occurring.

Given the extensive use of scanning technologies across NSW government, this case study, though now resolved, clearly demonstrates the importance of quality control in scanning processes, particularly for your high risk, high value business records.