Future Proof – Protecting our digital future

Peter and I like to get out and talk to people, to get a sense of how information is genuinely being managed out in different business environments.

Recently we have had conversations about digital information management and disposal with a group of people who regularly work on digital business operations with many different organisations.

We have also recently discussed digital disposal with the excellent members of State Records’ Digital Records Advisory Group.

Here are some of the key observations that we have taken from these conversations.

The group of people who regularly work with a lot of different organisations and across a lot of different digital business operations told us:

There is no awareness that there are significant information management problems in most organisations today

Most people in organisations have no idea that they have an information management problem.

Few business or ICT staff are aware that many corporate operations rely on having stable, long term, meaningful, accountable business information.

And few business or ICT staff are aware of the complexities, risks and challenges associated with creating and maintaining this stable business information in the diverse system environments that are used across organisations today.

There is no big picture view

The current state of digital information management, the number of records management silos and the diversity of systems across any one organisation means that there is no capacity for a consolidated view of information or customers or transactions across most organisations.

Solutions are increasingly being implemented to try and create a big picture view, but these generally have no information management within them

Many organisations are deploying ERP (enterprise resource planning) systems to build big picture views of information across an organisation.

Information life cycle management can be deployed as additional functionality within these systems, but the people we were talking to have never seen this done. Authorised disposal processes can also be applied but again, they have never seen this occur.

In organisations, the people deploying ERP systems are frequently unaware that the information within them needs to be actively managed. They are also not widely aware that some of the information in these systems needs to be kept very long term, that some can be routinely purged, and that strategic governance decisions need to be applied to:

• the use and administration of the system
• the design and management of information in the system
• the application of metadata in the system to build trustorthy and useable information
• the system decommissioning process.

Collaborative business partnerships involving records and information management staff are required to help business make informed decisions when changing systems or transitioning to ERP frameworks to prevent these significant business risks from occurring. Again however, the people we were speaking to have not yet seen records and information staff take part in these conversations.

There is no legacy data management

From what these people see of ERP or other system transitions, legacy data is almost always left behind.

If it is carried forward into a new system, it is only partial and often highly massaged so it can work within the new environment.

Business and ICT staff are generally unaware of the issues, risks, decisions they should make about legacy business information and consequently appropriate solutions are seldom designed.

There are no long term management strategies

The people we were talking to reported that the dominant storage environment they see for storing information that an organisation thinks it should keep ‘long term’ is the back up system.

There are no information management requirements in outsourcing arrangements

These people also see a lot of outsourcing arrangements. Because the organisating doing the outsourcing generally has no awareness of the information governance arrangements that should be applied to their business information and systems, they consequently issue no information management requirements or governance advice to their service providers.

The lovely members of the Digital Records Advisory Group told us that:

Language can be the key to building understanding about strategic disposal practices

It was noted that at the moment, language is a barrier to developing and implementing strategic, proactive digital disposal solutions.

In the digital environment, decisions about disposal need to be up front and strategic if information is going to survive for as long as it is needed by the business. In information management environments, disposal is increasingly becoming a very proactive process and the term is being used in many conversations with business and ICT staff to get them to engage with proactive and strategic information management decisions.

It was noted, however, that to ICT professionals, the word ‘disposal’ has a specific meaning to it, associated with very passive and end of life cycle processes and not at all strategic or proactive.

When communicating with ICT about the strategic outcomes that can be achieved through proactive disposal processes, it was recommended that information managers should be very alert to this meaning and consider different forms of language to convey their strategic intentions.

You do not necessarily have to keep everything, but you need to be strategic about how you do this and focus on business needs

Some members gave examples of how, instead of keeping large volumes of legacy transactional data, they have repurposed this data into new, useful, accountable forms of corporate information. For example, in some instances, large amounts of transactional data have been rolled up and aggregated into reports, such as monthly payments to vendors. These reports are then retained as corporate records, in place of the large amounts of transactional data that was used to compile them.

It was reported that this process has been important for transforming understandings of records as passive, legacy, unreferenced repository items to records as dynamic sources of valuable corporate information, insight, accountability and business intelligence.

This form of information transformation requires careful preparation to ensure the new corporate record offers all the same necessary information, evidence and accountability that the large volumes of individual transactional records provided. But provided safeguards are in place, it was reported that this kind of transformation can significantly simplify digital disposal challenges, and help demonstrate the worth of records and information management to the organisation.

It was also noed that implementing disposal decisions has to be focussed on meeting business specific needs.

For example, an organisation has a legacy system with millions of rows of metadata that can legitimately be destroyed but a few thousand of these rows, about 5% of the total, are needed to support the information of ongoing value in the system. An understanding of the business helped in this disposal process to ensure information with ongoing value could be retained and superseded information could be destroyed.

It was also noted that records and information staff may be some of the only staff in the organisation who recognise ongoing value in legacy information. It can become a complex and time consuming task to explain this ongoing value to business or ICT staff and to then arrange for the migration of the necessary information but if you have valid business reasons for your recommendations, and if you are recommending the migration of specific information and have appropriate plans for the destruction of time expired legacy information, the group had stories to share about the business value these types of interventions have contributed.

Start planning for migration even before you turn on your new system

Even with these successes behind them, the group agreed that no one ever considers data migration when a system is being designed and that this fundamentally need to change.

It was noted that people completely discount data migration at the start of system design and development and consequently ease of and a strategic approach to future migration is never considered.

This obviously creates significant problems for the management of high risk/high value information which generally has to outlive many individual technological contexts, but it also of course has major cost implications.

Some group members reported that in system projects they are aware of, data migration has accounted for up to 75% of total costs because of the complexities involved, and so planning for longevity and portability is clearly a significant issue that needs to be better managed.

Start planning for disposal now in cloud service arrangements

The group noted that a key scenario where legacy management is going to cause problems is cloud transitions. For example, when you need to change cloud platforms, how do you choose what to carry forward and what to dispose? Retention and disposal decisions clearly need to have been foregrounded in this scenario so that organisations have chosen a service provider who is able to export their long term value business information and repatriate it at a reasonable cost, and who is also able to accountably and systematically destroy the information the organisation no longer requires.

However while it was also noted that while replacing on premise services with cloud services was a key scenario, an even more complex scenario to manage from both a business and information management perspective is where organisations are adding a cloud solution on top of existing other corporate ways of doing things, with no means of federating or integrating the approaches. In these situations separate information silos can form, one on premise and one in the cloud, and good information management can be key to ensure governance, accountability and business useability can still be achieved in these diverse business frameworks. Good information management is also needed to ensure that all information needed for ongoing business purposes is fed back into corporate systems if the cloud services are terminated in the future.

Messages need to be about business and the information needed to support it, not the technology

Finally, the point was made that while technology can dominate discussions and approaches, when it comes down to it, conversations and strategies genuinely need to be about business and the information needed to support business.

The point was made that messaging must focus on the need to build governance frameworks around information in order to support business needs, and technology must not be allowed to become the focus. It will be necessary to develop strategies to support business information through its different  technological evolutions over time, but the focus must be on business and the governance arrangements that are needed to create and manage the information that the business needs through time.

Thank you very much to all those people who were so willing to share their knowledge, ideas and expertise with us. As always, we are very grateful!