Strategies for managing social media information – how do I capture social media records? February 8, 2013
As social media is relatively new and rapidly evolving technology, there are no defined, best practice ways for making and keeping social media records.
Organisations also use different social media channels, in a variety of different ways and for various different types of business operations.
Given there are no easy answers for how you should capture and keep your social media records, you should choose the strategies that best meet your business needs and technological environment, while making a full assessment of the potential risks involved.
Tools and approaches for social media record capture
The table below lists a range of current recordkeeping strategies for capturing and keeping your social media information, and the pros and cons associated with each approach.
It is likely that use of a combination of these tools will build the most effective social media information strategy for your organisation.
Tool/approach |
Pros |
Cons |
| Leaving data in its native social media application | No separate recordkeeping effort requiredNo additional software or applications requiredStaff using the application will know how to access and use the information it contains
Short term value information is not likely to be at risk
|
Long term value or high accountability information will be at risk due to projected frequency of system changeRisks to ongoing information accessibility exist because information remains under the control of an external third party Limited corporate information accessibility if a log-in is required to view or use data |
| Using available APIs to regularly export your information from social media applications | Information is regularly exported into corporate systemCorporate information is brought back into corporate controlExported information can be fed into all relevant business systems and processes
|
Technical knowledge is requiredAPIs can change and so API-based export strategies may require regular updateDifferent APIs will be required for each social media channel |
| Cloud-based back up system like Backupify | Free, basic online servicesSupport a wide variety of social media applicationsInformation can be regularly exported
Corporate information can be brought back into corporate control Some information is exported in open, non-proprietary formats (For example, Backupify generated Twitter reports are in PDF) These services work with a range of social media channels, including Google apps
|
These tools are back up tools, not recordkeeping tools.Their interfaces and data exports are designed to satisfy IT and backup requirements, not business or recordkeeping needs. For example, Facebook data is exported by Backupify in JSON which meets backup needs but which is not readable or accessible for standard business environments. With these tools, export of information out of social media applications is automated but downloading of this information out of the cloud is not. Downloading and capturing into corporate systems must still be performed as a manual, scheduled process.As they provide backup services, these tools generally download all your social media information, not information from a specific time period. The same legacy data will generally be downloaded each time you do a backup, resulting in significant amounts of duplication for high transaction accounts. |
| Cloud-based information services such as Social Safe and Archive Social | Free basic online services, scaling to fees between $US5 – $29 per year for full service optionsSupport a wide variety of social media applicationsConsolidate a variety of social media data into one exportable document
Information can be regularly exported into corporate systems and into corporate control
|
The long term accessibility of the data presentation format requires investigation |
| Social media monitoring or dashboard tools | Many third-party, cloud-based tools are available for free, such as HootSuiteAggregate information from several social media channelsCan piggyback information management needs on the back of existing reporting or monitoring arrangements
Provide reporting and listening services to monitor the effectiveness and impact of your social media presence Aggregating and reporting functionality creates very useful business information
|
Limited information accessibility if a log-in is required to view or use dataThere may be limits on the reporting and analysis information that is available for exportKey business intelligence and reporting information is contained in reports and analysis so exportability of this information needs to be verified
Some organisations use licensed applications such as Radian6 and Alterion. While offering good functionality, these systems can be very expensive to deploy and the exportability of their information needs to be verified.
|
| Reporting tools that come with your social media application, such as Facebook Activity Logs | Free, online servicesGenerally export to PDF and other widely open, accessible formatsCapture all activities that occur on social media sites
|
Export needs to be manually performedInformation is flat and not dynamically available to reuse or repurposeNeed to determine whether information exports can be performed periodically, or whether full information downloads are performed each time. |
| Analytic tools, like Google Analytics or blog software analytics | Free, online servicesUseful for monitoring blog use, search engine terms, referring sites, top posts and pagesSome information can be exported
Some corporate information analysis can be brought back into corporate control
|
May not export reportsMay not export reports in business-ready formatsMay need to be supported with screenshots, written reports or other ways of capturing the business information they contain. |
| General third party, cloud-based reporting tools like Storify | Free, online servicesCan allow you to gather a range of disparate social media information sources into one place to tell a storyParticularly useful for capturing different sets of online information about events or conferences
|
Export functionality may not be present |
| Use a purpose-built software tool | Can be designed to meet your specific business requirementsCan be designed to integrate and share information with your corporate business applicationsCan provide a very comprehensive and accountable recordkeeping solution
Information capture can be automated
|
Will take time and money to developWill possibly need to be upgraded as social media applications and approaches change |
| RSS feed | FreeUseful for a range of social media applicationsUseful for auto-populating Twitter and Facebook based on blog updates
Can send an email containing a complete blog post to a designated account Can be configured so that certain events such as a blog comment, retweet etc will trigger an email to sent to a designated account and this email can then be captured as a record
|
Emailed records will require manual intervention to capture into corporate systems for accessibility and useability |
| Excel spreadsheets manually updated with social media information | Records can be keptInformation can be kept in accessible formats | Information needs to be manually updatedInformation will require regular staff commitment to keep up to date For active accounts, there will be a significant cost in staff time
|
| Screenshots | Records can be keptInformation can be kept in accessible formats | Information needs to be manually updatedInformation will require regular staff commitment to keep up to dateInformation is flat and not dynamically available to reuse or repurpose
For active accounts, there will be a significant cost in staff time
|
| Reports of pre-scheduled posts | Record is compiled as part of process of developing and authorising social media posts | Information is not an exact representation of what was posted on social media sites but is an approved records of what was authorised to post |
Case study: Records staff can deploy social media monitoring toolsRather than rely on business areas to make and keep records of their social media activities, records staff can deploy social media dashboards and monitoring tools to monitor all internal social media accounts and to export the information from these accounts as required. |
Services in this area are very subject to changeIt is very important that you keep an eye on any recordkeeping solutions you implement for your social media. Many of the free online services that are available offer very good and useful functionality but these technologies are evolving rapidly. A solution that suits you today may evolve into something different tomorrow or a solution that exists today may disappear tomorrow. Many free services are experimenting with formats, functionalities and services and the capacities they offer may change quite regularly. All the free services are ultimately actually commercial operations and so the services they offer will be driven by business imperatives. For instance, Backupify announced in December that they will no longer be providing backups of LinkedIn data. Increasingly their corporate revenue is coming through backup for enterprise-based SaaS applications. They are therefore focussing less of their resources on consumer back up requirements and more on corporate requirements. Therefore be very aware of change and vulnerabilities in this space and monitor whatever services you deploy to ensure they continue to meet your business needs. |
Use recordkeeping channels that you already haveOne organisation, when seeking community consultation via social media, directed people back to its organisational blog to provide feedback. The blog was set up with an RSS feed so that any comments received on it were emailed automatically to relevant staff. Staff were able to capture these emails into their corporate records system, based on existing processes and procedures. |
Use management reports for recordkeeping purposes
Social media is still a relatively new technology in many government organisations. Management in some organisations is still yet to be convinced of the validity of social media as a business tool and, while its business relevance is being evaluated, many organisations have strong reporting requirements around social media.
If your organisation is unwilling to commit to third-party tools or other investments to support the management of social media information, business reporting can be captured and used as a means of making and keeping records of your social media operations.
| For example: Many organisations still plan their tweets and send their proposals through for official endorsement. These records could be captured as official records of your social media activity, rather than the tweets themselves. |
Use recordkeeping strategies to bring social media information together
You want to avoid your individual social media channels becoming separate silos of corporate information.
Thinking strategically about business and determining how specific business processes could benefit from the information generated by your social media channels can help to determine what information from your social media applications you should capture and share.
Third-party applications can help to automate this process.
Example: Social media dashboardsDashboards such as HootSuite offer the ability to bring together content from your variety of social media channels to build an overview of your social media presence. These tools can sometimes export consolidated social media data and this record can then be used as business intelligence in your organisation, as well as a formal record of your social media presence. |
Use social media recordkeeping as an opportunity to improve your business intelligence
You can use social media as an opportunity to know so much about your clients and your operations.
If you use monitoring or reporting tools to track or measure your social media operations, ensure that the tools you use can export copies of these assessments or reports.
These records can be captured and used across your organisation to feed into business intelligence, planning, service improvement etc.
You can use this reporting to:
- Measure the impact of the content you are sharing
- Identify topics of interest and relevance to your community
- Improve your services
Case study:
|
Specific information management issues relating to Twitter
Australian government Twitter statistics:
Source http://egovau.blogspot.com.au/2013/02/infographic-top-government-twitter.html |
Make use of Twitter hash tags
One way to gather together all tweets on a specific issue or service is to search on a specific hash tag.
Hash tags are a means of labelling and classifying tweets. Specific events such as a conference will have a hash tag (for example #ICA_2012), concepts can have a hash tag (#archives), or specific organisations or services can have a hash tag (#staterecords).
Searching for a specific tag will bring together all tweets that contain the hash tag which you can then copy and export.
Be alert to information loss that can be caused by short URLs
Many tweets contain links to longer forms of information and, because tweets must be 140 characters or less, many of these links take the form of short URLs.
Example: bit.ly/UpZXrR is the short URL for /what-recordkeeping-functionality-do-business-systems-need-to-provide/
Short URLs are likely to be less supported and accessible than regular URLs. They are designed as a short term service and no guarantees are provided about their longevity.
In addition, as you can see from the example above, a regular URL generally provides information about the location and context of an online resource whereas an auto-generated short URL does not.
In a social media recordkeeping strategy you may want to look at the short URLs contained in your organisational tweets or other social media sources. You should ask questions like:
- In a year’s time, is it going to be important to know what this information referred to actually was?
- Will we need to know what webpage a client was pointing to in their comment?
In many instances it may not be significant but, if in particular business areas it is going to be important to know what was being referred to, you should develop a means to capture a record of the referenced web page where necessary.
This capture could be a manual process to capture either the full URL or a screenshot of the page itself. Alternatively it may be possible to develop an automated process and deploy a tool that will automatically take a copy of the web pages referred to in your tweets.
Depending on their business purpose, some Twitter accounts will reference external links much more significantly than others. For example, a Twitter account used for marketing or communications will generally point off to a lot of other web-based resources, while another that is used to facilitate discussion and debate will not reference as many external resources.
Consider the use of different Twitter accounts for different business purposes
If it is likely that you will have a broad range of tweets and retweets in your account, ranging from business critical commentary through to tweets about new library acquisitions, you may want to create several Twitter accounts, rather than funnel all corporate business through one account.
If you have one account and some business critical information passes through it, if you use certain recordkeeping strategies such as running a report through a third party service, it will provide you with a record of all tweets in your account, not just the key tweets.
If you only want to capture key tweets, you can set up RSS feeds to provide you with copies of all your tweets, retweets etc and then you can capture the key tweets as emails and leave the others in Twitter.
Creating several accounts gives you more options for information management. For example, you might want to create a @CouncilXyoursay account where you encourage ratepayers to comment on current issues and a @CouncilXlibrary account for news and updates about the library. You could deploy different information management strategies for the different accounts, based on the levels of business risk you identify.
Specific information management issues relating to wikis
With these types of complex systems you need to ask fairly complex questions to try and determine exactly what parts of the system you need to keep as a record.
Questions to ask to help you define what records need to be captured and kept are:
- What information do we want and need to keep?
- Can we prioritise by risk and focus on extracting and maintaining critical data?
- What metadata is necessary to document context and for audit and management purposes?
- How long do the records need to be kept for?
- Do we need to keep change history and revisions?
- Do we preserve at intervals or at a particular project completion date?
- Do we preserve text content, or complete functionality, or the look and feel of the site?
- How do we capture records? What formats can or should be used for export and for ongoing storage?
Specific information management issues relating to collaborative editing tools
With collaborative editing tools like Google Docs, it is critical to consider the strategic management requirements that apply to this type of collaborative documentation that may be shared across organisations.
You need to develop rules or requirements such as:
- what technical and administrative controls are required, such as secure log in
- what version controls are required
- what revision history needs to be documented
- is there the capacity to lock down pages
- what back up services need to be implemented
- how will be the capture of necessary metadata be arranged
- how is ownership between multiple collaborators to be determined
- how will access arrangements between multiple collaborators be determined?
- when export will be performed – regularly or at project completion etc
amazing read! ….this has been very refreshing and informative even of social media professionals. And I am going to try out some of teh tools that you have mentioned. :) thank you for the post!
Hello Kate,
like always great content. I think this is one of the best and most practical orientated write ups on Social Media Recordkeeping I have seen so far and as you know we have been searching ;)
Given my very personal interest in this I wanted to point out a few things, that might get lost to a reader in all the intrinsic implications of records management requirements.
To my understanding none of the listed solutions really comply (or not without a lot of extra manual effort) with the current recordkeeping standards.
a.) To my knowledge all of them lack the necessary Metadata to comply with the minimum set as defined in “Module 2, ‘Make and keep full and accurate records”, especially for the business function/process and the record type.
This information would have to be applied manually to the record and given that this can change significantly from one tweet to another, that could take quite some work (e.g. importing the document multiple times, as different records etc.).
This metadata isn’t just important from a compliance perspective, but like anybody that has ever tried to find an email in their inbox from a few years ago knows, you either have a good filing system or a lot of time and luck on your hand.
b.) Secondly a critical component of a social media record capturing solution is the handling of the full extend of external links. As discussed already in another blog post a link can be interpreted like an attachment to an email. It often contains the key message of the whole conversation and as such is critical to capture. Sadly in the world wide web a link is a very fleeting thing and can change very quickly. Hence just capturing the link is pretty much unreliable especially for some of the records with longer term storage requirements (good luck following a link after 2 years, let alone 7 or more). To keep a proper record the organisation has to capture the content where the link pointed to as well.
There are certainly a few other points like storage in Australia etc., but to my understanding the two listed above are the key to core compliance.
I understand that currently one has to work with what is available and at least starting to capture is better than nothing. Let’s hope there is a good solution out soon *wink* ;)
Bora
Interesting article, but nothing on the legal aspect of archiving social media content ? It’s very important.
Hi Bora – thanks for your kind words and insightful comments. Yes, the issues you flag are really important considerations. With social media, it’s really important to think long term, think of desired functionality, think of what you will need tomorrow and the day after tomorrow to make best and most effective business use of your information. In your comments you are flagging these long term considerations – what are you going to need to make sense of, trust and reuse your business information now and in the future, and these are really important recordkeeping considerations. Thanks, as always, for sharing your views and experience. Kate
Hi David – thank you for your comment. It’s really interesting and I would love to have some more feedback from you on the specifics of what you mean. At State Records our key legal consideration is the State Records Act 1998 which basically says that government needs to keep its business information so that it can account for its actions and perform its business effectively. Our advice is designed to help government apply these State Records Act requirements to a social media environment where it can be very difficult to actually make and keep records. But there are a host of other legal considerations out there. It would be great to get your views on what other legal aspects of social media recordkeeping we should be considering and providing advice on. Cheers, Kate
Hi Kate,
Anil from ArchiveSocial here. First off, absolutely excellent work putting this guidance together. I am in the process of working through the full draft so that I can pull ideas for us to improve – so thank you! I will also be sure to send you comments in case there any areas in which conversely I might be able to add value.
I fully agree with Bora’s comments. In regards to metadata, there are actually two types of metadata we run into:
1) The complete, raw underlying data associated with a record that is not often apparent in the presentation layer (e.g. it doesn’t show up on Facebook.com or Twitter.com) . This is analogous to full the SMTP format when viewing the underlying source of an email message.
2) Additional information associated with the record for classification and long-term record keeping.
While we already preserve #1, the latter is something we would love to receive further guidance on. Our current plan is to support a simple tagging/note mechanism that can be used to annotate any tweet, post, photo, etc. A record would then auto-associate with the appropriate retention schedule based on its tags. I am curious if you would suggest other approaches/systems we should look at in regards to this type of metadata and classification?
Preservation of the URL in a social media post is another very interesting issue. Today we post-process all tweets to resolve short URLs to their full form, to ensure that the long-term record is not depending on a URL shortening service several years down the road. However, we don’t capture the content of the actual site that the URL references. We often look at email as our analogy, and current email archiving (or document archiving, for that matter) solutions don’t capture websites linked to either. That said, your suggestion that a link in a social media post is more really analagous to an email attachment does provide food for thought!
Once again, I appreciate work you’ve put into this guidance as well the the open exchange of information it has facilitated. As a vendor, we’ll do everything we can to be a productive part of this conversation and incorporate your ideas.
-Anil
One other comment: It would be great to see guidance in regards to how frequently an organization should think about capturing new content. If not using an automated solution, would you advise weekly, monthly, quarterly?
This is an important consideration given the realtime and dynamic nature of social media. It is especially important since social media largely consists of 3rd party content on 3rd party servers, which could be deleted at any time. I am curious to hear how think about this trade-off between effort and risk.
Hi Anil – Thank you so much for your comments so far on our guideline. I really appreciate your feedback. Providing further advice about metadata is something we want to do in the final version of the guidelines (due end of April). Possibly something we should also include is a bit of a ‘desired future state’ area where we could flag some of the technical developments that we would like to see, similar to those you discuss above. In the revised version of the guidelines we will also probably aim to provide more advice about the types of recordkeeping metadata that may need to be manually added to captured social media records, which may help to answer some of the questions you have. Thanks again for your feedback! Best wishes, Kate
Hello again! At present we talk about these types of decisions in terms of business risk – if the type of business you are using social media for is core, high risk business then your capture processes should be very regular as, as you say, there are risk in the technical infrastructure and because you may want to incorporate the information gathered quickly and regularly into business processes. In the guideline we are really trying to encourage people to think of social media information as active business information that should be (where relevant) fed as a contextualised, meaningful resource into corporate business processes and not just stored as a reference copy of communications for regulatory purposes. There is a very small amount of guidance (in /strategies-for-managing-social-media-information-determining-what-records-need-to-be-captured/) on the frequency issue but you are right, it really does need expansion. In the final version we will look to provide some more advice on the frequency of capture and give a few more examples of different scenarios. Thanks again for the feedback! Cheers, Kate