Cloud Storage and Digital Preservation: New guidance from the National Archives

13 May, 2014

So-called ‘cloud’ services are – appropriately enough, given the name – often vaguely understood in terms of both scope (what is a ‘cloud’ service exactly?) and appropriate deployment (What are they for? Should we be using ‘the cloud’ for research data?)  This situation isn’t helped by certain software companies and IT infrastructure providers slinging around the notion of ‘the cloud’ as a magic bullet to all your digital communication and storage needs. 

There is no such magic bullet and, as with all possible solutions, a large element of success is careful selection of the right tool for the job in your specific context.  Cloud-based solutions are no exception to this.  For organisations considering a cloud-based solution, help is at hand.  The National Archives has recently published guidance produced in conjunction with Neil Beagrie (Charles Beagrie), Paul Miller (Cloud of Data) and Andrew Charlesworth (Senior Lecturer in IT Law, University of Bristol).  This report promises an overview of the cloud as a tool for digital preservation and its usefulness for storage and access to digital assets.  The crucial considerations of legal, cost and security issues will also be covered, and current best practice will be illustrated by five new case studies (including one from the HE sector), showing a variety of approaches and cost models – this represents a valuable knowledge base from which the rest of us can learn.

The team behind the new guidance recently (13 May 2014) presented a useful online seminar which introduced the upcoming report, discussed some of the key issues around the use of cloud services and answered questions from the audience. 

A couple of the salient points for me were:

  • The National Institute of Standards and Technology (NIST) has a list of characteristics of 'cloud’ services, as follows: On-demand self service; broad network access; resource pooling, rapid elasticity and a model of measured service (i.e. the use of resources is metered and charging is on that basis). However, the team noted that other attempts at definition do exist; and that many service providers use the term ‘cloud’ about products and services that have only some – or even none – of those characteristics.
  • Cloud is not the answer to every IT or digital preservation problem. It may offer potential for repositories and archives but may need to be combined with other solutions.
  • Flexibility of cloud-based services allows relatively rapid and low-cost testing and piloting of providers.
  • There are a number of available cost models.  Some providers offer a set ten- or twenty-year preservation period for an up-front payment.  (This may be helpful to bear in mind when writing data management plans for some UK research funders who will only allow RDM costs incurred during the lifetime of the project.)
  • There is greater flexibility and more options in deployment of cloud services than before – public, private and hybrid models can work as useful tools for organisations of different sizes and with different requirements and budgets.
  • Thorough risk assessment is absolutely crucial – you must understand and assign the risks of any preservation solution, cloud-based or not.  There are legal implications here, too, in terms of how data security, IPR and the requirements of relevant legislation such as the Data Protection Act are handled.  Cloud solutions can offer some particular issues too; for example, the implications of data storage on servers located outwith the UK (and in the case of some providers, outwith the EU) must be fully understood and managed.
  • Scale can make a difference in negotiation – the bigger the market the more likely you are to get better costings.  Getting consortia together including working agreements is a particular effort but regional / national solutions may become very important in the future.

    NB: On that topic, it's worth noting that Janet has a framework agreement to facilitate the purchase of cloud and data centre services by Janet connected organisations. More information is available at

    https://www.ja.net/products-services/janet-cloud-services/cloud-and-data-centre-framework

  • Explicit provision must be made for defined exit strategies should the viability of your cloud provider change, or you need to move to another solution altogether.

The TNA guidance is available on their website at:

http://www.nationalarchives.gov.uk/documents/archives/cloud-storage-guidance.pdf

Also, here’s the relevant blog page about the work:

http://blog.nationalarchives.gov.uk/blog/cloud-storage-archives-match-made-heaven/

and those who are really keen can submit papers to the International Conference on Cloud Security Management this coming October – more information at:

http://academic-conferences.org/iccsm/iccsm2014/iccsm14-home.htm.

Do you use cloud-based services at your institution?  What have been the advantages or things to beware of?  Tell us in the comments!