Future Proof – Protecting our digital future

If you ask most people what is the single most difficult problem in managing information in their organisation, the response is likely to be related to the sheer quantity of records produced. Digital recordkeeping has certainly allowed us to create more and more records (and duplicates) than ever before.

Much of what is captured is also ‘unstructured data’ which is very difficult to manipulate and mine for business intelligence. Information technology specialists are introducing Information Life Cycle Management to try and manage unstructured data, but it is important for organisations to recognise the valuable contribution that records managers can make to ILM objectives.

Structured vs unstructured data

So what is unstructured data? Structured data is data that has already been interpreted. It is captured in a nice, orderly form, usually in clearly defined fields in a database. There are established tools and techniques to mine this information.

In contrast, unstructured data is created without these strict controls. Once created, it needs a human to interpret it as there is no clearly defined ‘structure’ for a computer to ‘understand’. When we create word processing documents, the body of email messages, presentations, multi-media files, audio, video, blueprints and web pages, we are creating unstructured data.
Structured data is often (but not always) created according to system specifications and pre-defined rules based on business needs. Unstructured data can multiply like mushrooms, and often those creating it have little or no understanding of whether it needs to be created, how valuable it is or how to manage it. That’s probably why it is commonly believed that 85% of all business information is in the form of unstructured data and why it continues to grow at a massive rate.

While it may appear to be cheap to keep adding new servers to store all the unstructured data an organisation creates, there are a number of hidden costs. For example, there are the costs of management and protection, the cost of staff time in locating and retrieving information or recreating the information that can’t be found. There is also the cost of migrating and preserving large quantities of unstructured data that don’t need to be retained and non-compliance or litigation costs arising from the inability to manage the data. These costs may not all be exclusive to unstructured data, but they are significant when considering the volume of unstructured data in most organisations. More efficient and desirable uses of valuable resources are required, particularly in times of economic crisis.

Information life cycle management as a solution

To manage unstructured data and reduce costs the information technology industry is embracing Information Life Cycle Management (ILM). ILM is not a just technology, it’s ‘a combination of policies, processes, practices and tools to align the business value of information with the most appropriate and cost effective IT infrastructure from the time information is conceived through to its final disposition’ (definition from Information Technologies and Information Storage industries, SNIA association).

ILM depends on organisational plans/policies outlining what data is required, how it is to be created and classified, how it is to be used over time and how critical it is to the business at any given point in time. This knowledge enables IT staff to determine where the data should be stored throughout its lifecycle and to automate migration of the data from one data store to another over time with minimal hands-on management. At the appropriate time, the right data can be archived to suitable storage and data that is no longer required can be destroyed.

ILM also involves disaster recovery and business continuity planning. Critical data is defined at the time of classification and processes such as mirroring or replication can be applied to remote arrays or sites.

How records managers can contribute to ILM

It is important to realise that ILM has been partly built on traditional records management practice, and that records managers are a valuable resource and should be included in the multi-disciplinary teams required to implement ILM effectively.

The key contribution that records managers can make to address the problems and costs resulting from the explosion in unstructured data is that records managers can advise organisations about what information must be retained and managed to meet their legal and business needs and regulatory requirements. They can also advise organisations about what information need not be captured at all, and what can be routinely destroyed.

In addition, the use of records management techniques can make the information within documents and other unstructured record types more discoverable and useable, via techniques such as the assignment of business-related metadata through registration and classification. This will ensure that critical connections to business value of the data are readily established.

In terms of the ILM process, records managers may:

• define what business, legal and regulatory requirements apply to business functions and processes
• advise what records should be created to support these requirements
• define essential metadata that should be captured to link the records to business, protect data integrity and support the management of records over time
• help to define metadata that can allow organisations to re-purpose and re-use business information
• create classification schemes linked with recordkeeping requirements
• advise which functions and processes and resulting records are critical to business operations
• advise when records are not required as frequently for business use (and therefore could be moved through suitable storage phases)
• advise how long records need to be retained in order to meet recordkeeping requirements
• advise when more ephemeral records and duplicates may be legally destroyed
• specify functional requirements for software to manage records and other information assets
• advise regarding security restrictions and access policies for records resulting from business functions and processes
• advise on recordkeeping aspects of migration and other preservation strategies for long term or archival records
• train staff members in how to meet recordkeeping aspects of information life cycle management objectives.

What all this means is that records managers can provide some of the essential information and support that will allow organisations to manage the uncontrolled growth of unstructured data and as a result operate more effectively and with lower costs.