Future Proof – Protecting our digital future

photo credit: fo.ol

Society today is producing far more information that it has the capacity to store and preserve. The gap between what we create and what will actually have the capacity to keep is also growing exponentially. ‘In fact, the production of digital information has already outstripped global server capacity by an estimated factor of four or five’ [15]. So say Daniel J Caron and Richard Brown from Library and Archives Canada in their disturbingly fascinating article in the current edition of the Canadian archival journal Archivaria (Number 71, Spring 2011), ‘The Documentary Moment in the Digital Age: Establishing New Value Propositions for Public Memory’.

Caron and Brown say that the world is in the midst of ‘an exaflood (of exabytes), the scope, scale and dimensions of which are overwhelming regardless of whose calculations one accepts, or the nature of the criteria being used, or in what year the measure was taken…Essentially the Web represents semantic and epistemological chaos from current public memory perspectives.’ [15] It’s a worrying but fascinating article! Caron and Brown argue that ours is ‘an information-laden and information-intensive society’ [4]. They say that we are in the midst of an amazing period of social, financial and business transformation driven by rapidly evolving capacities to create and use information. Information creation, processing and sharing have become so widespread in Western society that Caron and Brown cite Clay Shirky’s fantastic phrase – ‘here comes everybody’ [12].

And so, they ask, what does the volume of digital information out there mean for archives? Archives in this digital age are still very traditional organisations. We still operate using very traditional processes and procedures. The ‘documentary moment’ that Caron and Brown refer to in their title is the moment when a record is identified as an ‘archive’, as something that has long term value and meaning for the society that created it. In traditional approaches to archiving there has generally been a very long period between information creation and this ‘documentary moment’. Files traditionally sit on shelves for years before archivists assess whether they have continuing societal value and therefore whether they deserve to be classified and managed as ‘archives’. Caron and Brown argue that in today’s information society, the documentary moment is now. ‘In the digital world, the documentary moment is entirely active, strategic and present in the immediate ‘now’’. There is no period for reflection. Archivists need to decide now what is important. To set this decision aside for later discussion would mean the complete loss of valuable information, either through lack of planned preservation or its complete invisibility within the sheer mass of data being created and stored.

In the early 1990s brilliant Australian archival theorists like Sue McKemmish and Frank Upward began to question traditional archival approaches and came to the same conclusion. Their work led to the development of continuum theory and, as with Caron and Brown’s conclusions, from a continuum perspective there can be no remote, isolated, esoteric ‘documentary moment’. A record can and should be identified as an archive before it is even created and then managed appropriately to ensure its survival.

Continuum thinking was codified in the then Australian standard on records management, AS 4390 in 1996. This then influenced recordkeeping practice across Australia and internationally. But even though Australian approaches to recordkeeping have been informed by continuum thinking for nearly 20 years, we cannot be complacent, we also have to listen and respond to Caron and Brown’s warning.

This is because while the tools and processes may be in place in many environments to enable record creators to identify their records of long term value that require active management from creation, this identification is often not occurring. And, in Australia as elsewhere, the ground is shifting under our feet. Caron and Brown argue that traditional tools such as disposal authorities may no longer appropriately define the information universe that we need to deal with. Records are also rapidly evolving in the online world. Many of our processes and concepts are based on a paper paradigm but digital records are moving well beyond the strictures of their paper cousins. What is a record in a database, in a business system, in a participatory web 2.0 environment? The tremendous capacities for digital information re-use also problematise traditional tools and approaches as record value and utility can wax and wane through time and a record’s context and interrelationships can change. [9] Are our tools and approaches and definitions keeping pace with these changes? I don’t believe they are.

We are also yet to fully define and understand the true costs associated with archival storage in this digital world. Caron and Brown state that while digital storage containers are cheaper than ever, the costs of the ‘administration, management and accessibility of the information containers inside the storage containers…over time…[are] rapidly escalating out of sight’. [15-16] Should such economic realities be allowed to impact on our recordkeeping practices?

We live in an age of such information abundance, but also such significant information risk. This article raises brilliant thought-provoking, troubling, interesting and vital issues to consider. We would love any comments on how you feel these issues affect your records and recordkeeping practice and how you think State Records should be responding to these challenges. Let us know what you think!