Team Chaos have struck again! But do not fear, Team Digital Preservation are on hand, in what could be their funniest adventure yet.

Seamus Ross, Dean of the Faculty of Information at the University of Toronto (formerly Director of HATII at the University of Glasgow), said, ‘it was a tremendous opportunity to work with the CASPAR team and its Principal Investigator, Dr David Giaretta on developing this animation. As we have shown in earlier animations the medium of animation provides a positive way to introduce complex topics related to preservation.  Here the concept of representation information and the registry and repository of representation information (RRORI) both created by CASPAR and the DCC form the central element of this adventure.’

Dr David Giaretta, Principal Investigator on CASPAR (Cultural, Artistic and Scientific knowledge for Preservation, Access and Retrieval) and Associate Director of the Digital Curation Centre (DCC) commented, ‘representation information and the registry and repository of representation information (RRORI) respond to a core need in the digital curation and preservation communities for ensuring the long term understandability and usability of data. While the presentation of these concepts in this animation hides the complexity of many of the issues, it demonstrates the power and central significance of RRORI and the underlying concept of representation information.’

WePreserve is committed to raise awareness about digital preservation issues among the widest possible audience and to breaking down barriers to access.

To learn more about WePreserve please go to http://www.wepreserve.eu/

Post from: Future Proof - Protecting our digital future

Team Digital Preservation are back!

Most Commented Posts

• New IAMS contract can save you money
• Bridging from your CMS to your EDRMS
• Implementing EDRMS

Much of what is captured is also ‘unstructured data’ which is very difficult to manipulate and mine for business intelligence. Information technology specialists are introducing Information Life Cycle Management to try and manage unstructured data, but it is important for organisations to recognise the valuable contribution that records managers can make to ILM objectives.

Structured vs unstructured data

So what is unstructured data? Structured data is data that has already been interpreted. It is captured in a nice, orderly form, usually in clearly defined fields in a database. There are established tools and techniques to mine this information.

In contrast, unstructured data is created without these strict controls. Once created, it needs a human to interpret it as there is no clearly defined ‘structure’ for a computer to ‘understand’. When we create word processing documents, the body of email messages, presentations, multi-media files, audio, video, blueprints and web pages, we are creating unstructured data.
Structured data is often (but not always) created according to system specifications and pre-defined rules based on business needs. Unstructured data can multiply like mushrooms, and often those creating it have little or no understanding of whether it needs to be created, how valuable it is or how to manage it. That’s probably why it is commonly believed that 85% of all business information is in the form of unstructured data and why it continues to grow at a massive rate.

While it may appear to be cheap to keep adding new servers to store all the unstructured data an organisation creates, there are a number of hidden costs. For example, there are the costs of management and protection, the cost of staff time in locating and retrieving information or recreating the information that can’t be found. There is also the cost of migrating and preserving large quantities of unstructured data that don’t need to be retained and non-compliance or litigation costs arising from the inability to manage the data. These costs may not all be exclusive to unstructured data, but they are significant when considering the volume of unstructured data in most organisations. More efficient and desirable uses of valuable resources are required, particularly in times of economic crisis.

Information life cycle management as a solution

To manage unstructured data and reduce costs the information technology industry is embracing Information Life Cycle Management (ILM). ILM is not a just technology, it’s ‘a combination of policies, processes, practices and tools to align the business value of information with the most appropriate and cost effective IT infrastructure from the time information is conceived through to its final disposition’ (definition from Information Technologies and Information Storage industries, SNIA association).

ILM depends on organisational plans/policies outlining what data is required, how it is to be created and classified, how it is to be used over time and how critical it is to the business at any given point in time. This knowledge enables IT staff to determine where the data should be stored throughout its lifecycle and to automate migration of the data from one data store to another over time with minimal hands-on management. At the appropriate time, the right data can be archived to suitable storage and data that is no longer required can be destroyed.

ILM also involves disaster recovery and business continuity planning. Critical data is defined at the time of classification and processes such as mirroring or replication can be applied to remote arrays or sites.

How records managers can contribute to ILM

It is important to realise that ILM has been partly built on traditional records management practice, and that records managers are a valuable resource and should be included in the multi-disciplinary teams required to implement ILM effectively.

The key contribution that records managers can make to address the problems and costs resulting from the explosion in unstructured data is that records managers can advise organisations about what information must be retained and managed to meet their legal and business needs and regulatory requirements. They can also advise organisations about what information need not be captured at all, and what can be routinely destroyed.

In addition, the use of records management techniques can make the information within documents and other unstructured record types more discoverable and useable, via techniques such as the assignment of business-related metadata through registration and classification. This will ensure that critical connections to business value of the data are readily established.

In terms of the ILM process, records managers may:

• define what business, legal and regulatory requirements apply to business functions and processes
• advise what records should be created to support these requirements
• define essential metadata that should be captured to link the records to business, protect data integrity and support the management of records over time
• help to define metadata that can allow organisations to re-purpose and re-use business information
• create classification schemes linked with recordkeeping requirements
• advise which functions and processes and resulting records are critical to business operations
• advise when records are not required as frequently for business use (and therefore could be moved through suitable storage phases)
• advise how long records need to be retained in order to meet recordkeeping requirements
• advise when more ephemeral records and duplicates may be legally destroyed
• specify functional requirements for software to manage records and other information assets
• advise regarding security restrictions and access policies for records resulting from business functions and processes
• advise on recordkeeping aspects of migration and other preservation strategies for long term or archival records
• train staff members in how to meet recordkeeping aspects of information life cycle management objectives.

What all this means is that records managers can provide some of the essential information and support that will allow organisations to manage the uncontrolled growth of unstructured data and as a result operate more effectively and with lower costs.

Post from: Future Proof - Protecting our digital future

Future Proof Update November 2009 – Controlling the growth (and costs!) of unstructured information in your organisation

Also of interest:

• Future Proof update March 09 – Using ROI metrics to justify digital recordkeeping project spending

•  How would you describe your change management approach, strategy and implementation plan, what were the major elements (i.e. communications, training etc) of the strategy/plan?
• What % of your system purchase and project costs were change management (change management equals all leadership, sponsorship, communication, training, coaching, other support, restructuring, job redesign, new roles, allowances and change management resources such as change managers, trainers etc)
• What would you do differently if you were designing and implementing your project again?
• What did you think you did well?
• Any other lessons learned?

Post from: Future Proof - Protecting our digital future

Change management in an EDRM project

Also of interest:

• Future Proof Update August 2009 – Selecting software for managing records
• Selecting recordkeeping software to suit your organisation
• Future Proof update Feb 09 – Managing digital records guidelines

However, as servers are being created smaller and faster, they are also using more and more energy. The 2006 Worldwide Server Power and Cooling Expense: 2006-2010 Forecast report by IBM Data Centre estimated that in 1996 a small to medium sized server required 150 watts of power and approximated that by 2010 this will have increased to 450 watts. The report also estimated that in the year 2000, for every dollar spent on new servers, 21 cents was spent on power and cooling. By 2010 it is predicted that for every dollar spent on new servers will require 71 cents to be spent on power and cooling. Whilst servers and hence data storage may be considered cheap now, the associated costs of power and cooling, including environmental costs are increasing.

Simple measures that can limit power usage include:

• Ensuring airflow is sufficient – In general, unrestricted air flow requires less power for cooling efforts.
• Dust and dirt – Dust can clog equipment and cling to the sides of active equipment. There should be an active program for the regular cleaning of the entire digital repository.
• Moving more cables into overhead raceways opens up airflow underneath floor panels.
• Only use lights when necessary or use a motion control or timer – lights use electricity and generate added heat which requires cooling.
• Reduce overall data centre lighting by using portable lights within each cabinet. This also helps technicians as light can be adjusted and directed where it is needed.

Each small adjustment results in less power and cooling and should be an aspect of the design plans for a digital archive. As NSW and Australia move towards reducing their carbon pollution, it is imperative that all digital repositories whether public or private are aware of the environmental impacts as well as the potential expense of their endeavours and aim to minimise their energy consumption.

Support climate change www.blogactionday.org

Post from: Future Proof - Protecting our digital future

The Green Digital Archive

Also of interest:

• Future Proof update June 2009 – Is your digital information at risk?
• Future Proof update Feb 09 – Managing digital records guidelines
• Pressing delete

Many of us can feel threatened and anxious when new ways of working appear on the scene, expecially when they are associated with technology which we may not fully understand. However the flipside of this is to recognise that when any new technology is introduced into your organisation, it is an opportunity to demonstrate the value of recordkeeping advice and how it can help with managing risk.

How?

Let’s imagine that your organisation has set up its own Twitter account. The purpose of it might be similar to Mosman Council’s, that is, to inform constituents of what’s happening in Mosman (see http://www.mosman.nsw.gov.au/web/external/twitter).

Or like the NSW Police it might be to report news and events that might affect the public or seek assistance in investigations (see http://twitter.com/nswpolice).

Just from these two examples it is clear that this technology can be used for purposes that have different degrees of risk associated with them. In each case, a recordkeeping professional can advise on matters like:

• What recordkeeping requirements exist for these types of communications? ASre they going to be regarded as formal or informal communications and what risk would be associated with not keeping them as records?
• How long do they need to be kept?
• What other records are being kept that document this kind of interaction with the public and how can we relate them?
• What records of the establishment and management of the Twitter account should be kept?
• What are some options for routine capture of Tweets to an official recordkeeping system?   
• What information has to be kept with the Twitter records (Twecords?) to ensure they can be found and used (metadata)

Answering these and other questions will be an important part of establishing a set of rules and practices for the use of the Twitter account, which would also address matters such as privacy, complaints and appropriate use.

Getting a seat at the table

..is not always easy, especially when the records team is traditionally left out of ICT driven initiatives.

One way might be to offer a practical, low cost solution such as one of the free tools explained in Charnita Fance’s blog posting ‘How To Backup Your Twitter Archive’ posted on MakeUseOf.com (http://www.makeuseof.com/tag/how-to-backup-your-twitter-archive/ – thanks to Katharine Stuart for finding this!) These include TweetBackup – which allows you to export up to 3,200 tweets in RSS, text or HTML – Tweet XML or Tweettake.

For further information on managing records of web 2.0 business, check out our guidelines Records management and web 2.0

Post from: Future Proof - Protecting our digital future

Keeping records of your web 2.0 business

Also of interest:

• Future Proof Update July 2009 – Technology and the changing nature of records management
• Future Proof update April 2009 – Records management and web 2.0
• Bridging from your CMS to your EDRMS

As you’re probably aware, a number of software products whose specific purpose is managing records and other information assets are available in the marketplace.

The NSW government has assessed a wide range of these products and selected a small number that best meet the challenge of managing the State’s information assets. The chosen software solutions are identified in Contract 2602 GSAS Information Asset Management Systems (IAMS) Software Applications, discussed in detail at State Records’ Future Proof site [http://futureproof.records.nsw.gov.au/selecting-recordkeeping-software-to-suit-your-organisation/]

In summary, what is offered by the contract is a range of different solutions selected to meet a range of different needs. This type of approach has been adopted because information management abilities, capacities and needs differ significantly across government organisations. Some organisations already have good levels of recordkeeping compliance and are looking to progress their information management beyond the basics into increased levels of automation, web content management and knowledge sharing. Consequently the contact lists specific vendors who offer specially chosen software packages and combinations that will enable these advanced levels of information management to be achieved.

In contrast, a large number of organisations are still working hard to achieve minimum levels of recordkeeping capacity. They are working to get the basics of records control, classification, use and disposal right and so the contract therefore also offers a range of vendor and software solutions to enable these critical outcomes to be achieved.

To work out how the solutions listed on the contact can best help your organisation, the Government Chief Information Office has developed a tool you can use to assess the current state of your organisation’s information management and to identify where you’d like to move to in the future. Based on the input you provide to the range of specific yes/no questions, the tool, called the Information Asset Management Maturity Matrix and Assessment Tool, then calculates which information management solutions available on the contract will best meet your current and future needs.

Remember, it’s horses for courses

It’s important to note that not all products available on the contract will suit all environments. Vendors and solutions have been chosen for their ability to meet particular needs. Some were chosen because they have highly developed solutions that are appropriate for those organisations with advanced recordkeeping and information management capacities. Others are much simpler solutions, designed for environments with more minimal requirements. So it’s really important to know your environment, know your requirements, use the assessment tool and choose a solution that best works for you.

The process of assessing the software solutions was massive…

All solutions on the contract had to measure themselves and report against a massive functional requirements statement developed by State government experts.

The functional requirements statement covered 18 specific areas of information asset management – records and document management processing, user and identity management, document relationship management, storage management, security and access management, search and retrieval, reporting, audit management, email management, content management, content input/output management, workflow, content quality and organisational controls, content rights management, web content management, collaboration and productivity, knowledge management and operational and technical support. In total, there were 1547 lines of detailed requirements specified.

Because the functional requirements were so extensive, a number of the solutions proposed are combinations of software which, when implemented together, offer a comprehensive solution but which, as separate products, may not meet your specific needs. Where products are bundled together on the contract it’s important to realise that these products have intentionally been combined and implementing one without the other runs the risk of not meeting your stated recordkeeping needs and requirements.

What about SharePoint?

There has been a lot of discussion in records management circles about Microsoft’s Sharepoint software, with many records managers being asked why they should not implement it for all their information asset management needs including records management. This product is listed on the GSAS Contract but it is only listed as a component of a specific software combination and it should not be regarded as a stand-alone option for providing records management capabilities. Under the terms of the contract Sharepoint on its own is not approved for use for basic records management; that is, it was not approved for use in Levels 1A&B (Basic physical / electronic document / records management, Level 2 (Document tracking and user/security management and Level 3 (Advanced workflow and administrative controls).

SharePoint is on the contract because it offers excellent abilities in workplace collaboration and is useful for web content management. When it is combined with the very comprehensive TRIM records management software system (as it must be under the terms of the contract) it helps to provide a solution that can address all 18 of the areas specified in the functional requirements. 

Which vendors are on the contract and why should I use them?

The following vendors have been appointed to Contract 2602: GSAS Information Asset Management Systems Software Panel:

• Alphawest Services (TRIM + Microsoft SharePoint Server)
• ELO Digital Office
• EMC (Documentum)
• Interwoven
• Netcat
• Objective
• Opentext
• Oracle
• Tower Software Engineering (TRIM)
• Vignette

Using the contract allows you to purchase software that you know as been comprehensively assessed and will give you significant cost advantage, due to the government’s buying power. So use the assessment tool, work out your own specific needs and requirements and implement the solution that is going to work best for you.

Share your IAMS knowledge and experiences!

The Government Chief Information Office has set up an online information sharing and collaboration site to support the GSAS IAMS contract that NSW public sector staff, including recordkeeping professionals, are invited to join.

The purpose of the Information Asset Management Systems Knowledge Resource Centre (IAMS KRC) Group is to enable the NSW Government to take advantage of the collective knowledge and skills, IAMS implementation experience and intellectual capital of members and their organisations. To do this the KRC will provide for a discussion on topics of interest, a repository for material or relevance, and areas to collaborate on projects of interest.

If you are interested in joining the IAMS KRC contact Peter Clare peter.clare@commerce.nsw.gov.au

Post from: Future Proof - Protecting our digital future

Future Proof Update August 2009 – Selecting software for managing records

Also of interest:

• Change management in an EDRM project
• Future Proof update Feb 09 – Managing digital records guidelines
• Selecting recordkeeping software to suit your organisation

Steve Bailey, in his book Managing the Crowd: Rethinking records management for the web 2.0 world (Facet Publishing, London, 2008) argues that traditional records management as we know it needs to evolve in order to meet the needs of the web 2.0 world. Factors driving this change include the increasing breadth, complexity and volume of digital information that we are creating and using, the control and management issues raised by cloud computing, and changing behaviours in relation to information, as evidenced by the popularity of social networking, folksonomies and user reviewing. Bailey asks: how can we harness this change for our own purposes? For example, how can records managers learn from the fact that many users will tag a resource on the web but are unwilling to enter metadata in an EDRMS?

Bailey proposes a radical rethink of the role of users in recordkeeping systems. He suggests allowing users greater ability to describe, review and even rank information resources based on their usefulness and relevance, enabling users to suggest access levels and contribute to the decision making process about the retention of information. To continue to achieve our information and records management goals, Bailey argues, we need to find ways to make records management tools popular in the same way that online services such as the bookmarking site Del.icio.us are popular – by adopting their functionality, format and style. In this way, he argues, records managers can both make the recordkeeping system a tool that users ‘love and can’t manage without’ and can take greater advantage of users’ in depth understanding of the information they create and use every day.

Understanding users’ information requirements and preferred ways of working will be key to succeeding with more user-centric recordkeeping strategies. Methods such as surveys, observation and interviews can allow you to gather information on your users to help improve your recordkeeping systems and processes to ensure they are appropriate and effective in a web 2.0 world.

Post from: Future Proof - Protecting our digital future

Future Proof Update July 2009 – Technology and the changing nature of records management

Also of interest:

• Change management in an EDRM project
• Keeping records of your web 2.0 business
• Future Proof Update August 2009 – Selecting software for managing records

It is interesting to see the following advice in the article from digital forensics expert Graham Thompson:

“The trick in exposing what’s fake or real, Thompson explained, is to obtain an electronic copy of the email and look at the ‘interrnet headers’.

Thompson later goes on to say:

“I mean, you’d ask your advisers, show me some more proof, show me other things, not base it all on one piece of flimsy evidence that could be something totally different to what it is.”

In other words, check the metadata! This means both the metadata showing the electronic provenance of the message and metadata applied for recordkeeping purposes when an email is registered in a recordkeeping system. Both are vital for proving the email’s authenticity as a true record of what was communicated.

To learn more about capturing authentic email records that will stand up under scrutiny, go to our short guide Recordkeeping in Brief 49: FAQs Email and recordkeeping.
To find out more about metadata for records, go to Recordkeeping in Brief 18: Introducing recordkeeping metadata.

Post from: Future Proof - Protecting our digital future

What can we learn from Utegate?

Also of interest:

• Future Proof update Feb 09 – Managing digital records guidelines
• Future Proof update May 2009 – Email overload! Time to tame the monster
• Pressing delete

• loss of authenticity – if, for example, you cannot show that the record has been kept tamper-proof or if records management actions like transfer or migration of the record were not properly authorised
• becoming unreadable – as a result of relying on a piece of software that has become outdated or which a vendor has stopped supporting
• being lost altogether – because of a computer virus, ‘bit rot’ or damage to the medium it resides on.

Some of these problems are highlighted in an entertaining way in a recent video made by Digital Preservation Europe following the adventures of ‘Digi-Man’ in his quest to rid the world of threats to digital information. You can view the video here (requires access to You Tube).

So what steps can you take as to protect against these types of threats to your digital records? Here are some strategies that you can start implementing today:

1. Keep digital records in recordkeeping systems – when digital records are registered and tracked using recordkeeping metadata, they can be shown to be authentic, reliable records if challenged. Check the Standard on digital recordkeeping for the minimum requirements relating to recordkeeping metadata.
2. Use a limited number of formats to create records, preferably open standards – by using formats like ODF (Open Document Format), your information is less at risk from software companies changing their systems making information created in older versions of their products unreadable
3. Try to avoid putting digital records on removable media- ‘Out of sight out of mind’ means that one of the most common reasons behind digital information becoming unreadable is that it has been sitting on an old floppy disk or some other medium in a drawer for years. Also by keeping the records online they will be protected by your organisation’s virus protection and back up processes
4. Take care when migrating records – this can be one of the riskiest times in a digital record’s life! Our Managing digital records guidelines contain detailed information on managing the migration process.

Post from: Future Proof - Protecting our digital future

Future Proof update June 2009 – Is your digital information at risk?

Also of interest:

• The Green Digital Archive
• Future Proof update Feb 09 – Managing digital records guidelines
• Pressing delete

The good news is you are not alone!

NSW government organisations receive millions of emails each year, with an estimated 40% – 80% of these related to official business. State Records has estimated that government organisations only capture between 0% and 40% of business emails. An estimated 119 million emails are not being captured each year.

The bad news is there is no silver bullet for this monster of a problem. State Records often receives enquiries regarding email management and our advice is always the same. It is the responsibility of all NSW government organisations to ensure that email is appropriately dealt with as a record.

State Records advises that to effectively manage your corporate email there are three requirements necessary. These are:

• An appropriate technical solution in place
• Clear policy and procedures
• Ongoing staff training in email management.

Almost all email – related enquiries relate in one way or another to the above requirements.

Technical Solution

An appropriate technical solution will enable people to place emails in the correct location and remove the need to use email vaults and back up tapes.

Policy and procedures

Clear policy and procedures help to identify what needs to be captured, who should capture and when emails should be captured.

Ongoing training

Staff training encourages correct practice.

All organisations have different cultures. Some will enthusiastically jump on the email and records management train while others will prefer to leave the problem to someone else or worse – ignore it. No matter the organisational culture, the quicker you begin to deal with emails, the faster you will see results and the problem will begin to reduce in size. Slowly people will begin to change their practices and the monster within that is email mismanagement will be tamed.

If you have any suggestions or helpful hints to your fellow ‘Email tamers’ please let us know! The Future Proof blog is there for your use.

State Records recommends that anyone trying to manage emails review the email management section in Managing Digital Records guideline as well as looking at Recordkeeping in Brief 49: Emails and recordkeeping.

Post from: Future Proof - Protecting our digital future

Future Proof update May 2009 – Email overload! Time to tame the monster

Also of interest:

• What can we learn from Utegate?
• Future Proof update Feb 09 – Managing digital records guidelines
• Pressing delete