ADS and the Data Seal of Approval – case study for the DCC

ADS and the Data Seal of Approval – case study for the DCC

Introduction

The Archaeology Data Service (ADS) was established in 1996 to “support research, learning and teaching with high quality and dependable digital resources”. To ensure the long term sustainability of these resources the ADS instituted a migration-based framework for a digital archive. Over the course of the intervening 15 years the digital archiving landscape developed quite dramatically, during which time we too have matured in the way we undertake our archiving, adopting new standards and guidance as they evolve.

The first well defined digital archiving standard we adopted was the Open Archival Information System (OAIS) Reference Model. This is a framework which defines the mandatory responsibilities of a digital archive and maps out the core activities (and relationships between them) that need to be carried out in order for the system to work. The OAIS model includes a road map for the development of related standards, one of which relates to the accreditation of archives. However, while the OAIS standard itself introduces the concept of conformance, it has never been something which has been easy to define or measure.

Trusted Repositories Audit and Certification

When undertaking to curate data for the foreseeable future (and beyond) the concept of ‘trust’ is of paramount importance. Yet in a young discipline such as digital archiving, it is very difficult to demonstrate the potential for longevity of curation. So when the notion of a ‘Trusted Digital Repository’ appeared about 5 years ago it was immediately something with which we wanted to engage. When the Trusted Repositories Audit and Certification (TRAC) checklist was published in 2007 we were interested to find out how we stood up against its criteria. The TRAC checklist, which includes over 80 points with which to comply, enabled us to establish a benchmark against which we could continue to develop and test our internal policies and procedures.

The checklist is very detailed. The 84 criteria cover every element of the digital archive, from examining the overarching organisational framework to scrutiny of the smallest detail, such as checking the authenticity of individual files within the archive. In 2007, when the ADS undertook an internal exercise in comparing our policies and procedures with the TRAC checklist, we were successful in fulfilling over two thirds of the criteria. While we felt that this result was a positive one, it did highlight areas of practice that needed some improvement and we are continuing to work towards achieving those goals. While this period of self examination has been illuminating, there is still no formal certification framework for the TRAC checklist in the UK, and we were keen to follow other certification methods when they became available.

The Data Seal of Approval

The Data Seal of Approval (DSA) guidelines were initially drafted in 2008 but it wasn’t until 2010, when the guidelines and assessment process became more firmly established that the ADS considered making an application. One of the things that attracted us to the DSA was the spirit of openness and trust on which it is built; reflecting the nature of the relationship we engender with our data depositors. The repository to be certified writes their self-assessment on the basis of the 16 DSA guidelines, providing references to relevant documentation that demonstrates or exemplifies compliance with each point. Documentation referenced within the assessment should ideally be available to the DSA board for peer review, and also, in the spirit of the assessment, to the wider public. The repository is not visited by the reviewer.

How to do it:

A copy of the Assessment Manual can be downloaded from the DSA website:

The Assessment Manual includes details of each of the 16 guidelines, the minimum requirement for each guideline (which are scored between 0-4) and some guidance notes for the applicant which describe the sorts of issues you might wish to address. Of the 16 guidelines, three relate to the data producers, ten to the quality of the data repository itself, and three to the use (and users) of the digital data within the archive. Once you have read the assessment manual, it will become apparent which policy and procedure documents you will need to reference as part of your assessment. You may need to carry out some work on writing, reviewing and making available documentation that relate to your archival procedures.

The submission and application process is carried out on-line via an electronic form on the DSA website. Once your documents are in order, the first stage in beginning  your self-assessment on the DSA website is to go to the application tab and fill in the name and address of your repository and your own contact details (as the main applicant). Submitting this form will enable the DSA team to initialise your application. You will be e-mailed your login details with which you can access the on-line assessment form. The form does not have to be completed in a single sitting.

For each of the 16 guidelines you must score your repository using a number between zero and four (zero being ‘Not Applicable’ and four being ‘This guideline has been fully implemented for the needs of our repository’). You must then describe briefly how you carry out the relevant activity (or procedure), and provide links to any pertinent documentation in support of your description. Your descriptions can be brief and to the point, although some topics may require more by way of explanation.

An example of one of the DSA Guidelines with our self-assessment below should give an idea of the level of detail and volume of evidence required:

You do not need to get full marks (4 out of 4) for every guideline in order to pass the assessment phase of the DSA. There is a minimum requirement specified for each guideline, each clearly stated within the DSA assessment documentation. In addition, there is also the option that you may outsource certain repository functions to another DSA approved archive. It is clear from the documentation available which of the guidelines can be outsourced. For example, the ADS outsources some of its deep storage preservation functions to the UK Data Archive, who have also been awarded a DSA.

Once you are happy that you have completed each of the 16 sections of the form, you may submit the form for approval to the DSA board. They will allocate a reviewer, who will be a member of the DSA board or another qualified individual (depending on the subject area of the repository).

The peer review process:

The reviewer is given a time frame of 2 months to assess each application. They will check how you comply with each of the guidelines and also look at the documentation provided to ensure that they trust that your curation activities are appropriate and meet their standard.

Within the on-line form there is the facility for the reviewer to accept or reject your self assessment of each of the guidelines and follow that up by making suggestions and comments or asking questions on areas that are not clear. The repository then has the opportunity to go back into the form and make or implement changes before resubmitting.

Where the application is successful, the repository will be awarded a Data Seal of Approval. They will then be able to use the appropriate DSA logo on their website and publicity material and will appear in the list of accredited organisations on the DSA website. The self-assessment (with reviewer’s comments) will be available from the DSA website for public scrutiny.

Time taken:

Application for the Data Seal of Approval is a fairly straightforward process and may not require a great outlay of time. Time taken to apply for the DSA is dependent on a couple of factors:

1. How far you are from meeting the 16 criteria – you may need to introduce changes in order to pass and these could take time to implement
2. How well documented existing policies and procedures are

The process of applying for the DSA for the ADS took approximately 4 days of staff time. A day was needed to work through the self-assessment form and record how we met each guideline (consulting with colleagues where necessary), and the rest of that time was spent updating existing documentation and making it available on-line. Once comments were received back from the reviewer it did not take very long to address them and make a few minor changes to the self-assessment in order to complete the DSA assessment process.

The amount of time required to go through the application process for other organisations will vary depending on their current situation. For the ADS, all our documentation was in existence in some form, it just needed reviewing, in some cases updating and surfacing via the website.

Issues for the ADS:

In the spirit of the openness the DSA recommends that the main policy and procedure documents that define the internal workings of your archive should be accessible both to the reviewer and to the world at large. For the ADS most of the policy documents that we required as proof that we had addressed the 16 guidelines were in existence, however, some had only existed as internal working documents rather than formal policy or procedural statements. Transforming the working manuals into documents suitable for mounting on line was the most time consuming element of the assessment process. 

Even if we had not made an application for the DSA, the process of updating and making public our documentation would has been a worthwhile exercise. The DSA application has coincided with the release of a new website for the ADS. Because of the DSA work we carried out, we have been able to introduce a new Preservation section to our website where we can make key preservation manuals and policies available:

Though it has never been a deliberate or calculated step, in the past our public-facing side has been heavily focused on the resources we have archived and the data on our website, rather than an explanation of ‘how we do archiving’. Information about the processes that go on behind the scenes within the archive itself has been, hitherto, hidden from view and may have been considered a ‘dark art’ by those not directly involved in digital preservation work. By making our preservation manuals publicly accessible we hope to engender greater knowledge exchange between ourselves and other archives and also strengthen the confidence that data depositors feel.

Another issue that we faced was that there were certain documents that we deliberately did not want to make public. Our Disaster Recovery Plan is a case in point. This document is one which has been in existence internally for many years, but due to the sensitive nature of some of the information within it (staff home telephone numbers; information on how to access server passwords) it is a document that we were not prepared to publish to a wider audience. However, the reviewer understood these issues and was happy with the level of documentation that we had made available.

Benefits:

There were two primary reasons for the ADS to apply for the DSA. Put simply, we wanted to reflect on our own performance and we also wanted to be able to demonstrate to our peers and user base that we were a trustworthy repository for their data. But there are others too:

• Internal Review: Undertaking the work toward the DSA enables us to reflect on the way we work and review our policies and procedures. In such a fast changing sector it is always beneficial to take time to consider whether we continue to meet the standards as they develop. We believe that the work we have undertaken in complying with the DSA guidelines will form a firm foundation should we wish to apply for other types of certification (e.g. TRAC) as and when those routes become available.

• Establishing bona fides:  We believe that having the DSA will enhance our reputation both within our designated subject based community (archaeology) and within the wider world of digital preservation. The DSA will be a useful benchmark for comparison with other archives. This is of ever increasing importance as there are several different interpretations and definitions of ‘digital archiving’. The DSA supports a very specific interpretation of digital archiving which moves us away from interpretations which involve simply allowing on-line access or filing a CD in a drawer.

• Enhancing the trust of our users: Digital archives, by their nature, do not benefit from having a long track record of sustainability unlike their counterparts in the world of traditional paper archives who have built up trust with both users and depositor, sometimes over several centuries. The DSA enables us to demonstrate to both the users of our archive and depositors to our archive that we are working to a set of standards and have been judged to have met those standards.

• Building a community: We believe that the wider digital preservation community will benefit from engaging in this process.  The fact that the UK Data Archive has been awarded the DSA too is of direct benefit to us. Our long–standing relationship with them as the providers of our deep storage and backup facility could be seen to have been ‘ratified’ or approved via this process. Gaining the DSA embeds us within a community of archives working to higher standards and potentially allows us to benefit from closer ties and relationships with them. It opens up possibilities of working with others to enhance our policies and procedures.

Renewing and reviewing:

The ADS have been awarded the 2010 Data Seal of Approval. This means that the DSA guidelines our archive has been assessed against are the guidelines that were finalised in June 2010. This award will last indefinitely and will not be revoked. The DSA board are aware, however, that they may need to amend the guidelines in the future, or increase the minimum requirement score for a guideline (some of the guidelines currently have a score set as 2 or 3 but there is an intention to increase all of these to 4 by 2015). When new and updated guidelines are made available, all repositories currently holding a DSA will be automatically informed and given an opportunity to apply for the latest version of the DSA against these new guidelines.

There is undoubtedly room for improvement in every digital archive and the hope is that this rolling system of DSA renewal will encourage archives, including the ADS, to continue to make improvements and strive towards better policies and practice in the curation of digital data.