Web Curator Tool

The Web Curator Tool (WCT) is a workflow management application for selective web archiving. WCT allows users to target websites that they wish to include in their collection, create and manage schedules to automatically harvest those sites, and package the collected files to easily submit them to a digital archive.

Provider

Developed by the National Library of New Zealand and the British Library, initiated by the International Internet Preservation Consortium. Currently maintained by Oakleigh Consulting Ltd.

Licensing and cost

Apache Public License – free.

Development activity

Version 1.6 is now available on Sourceforge and was released on 5th Dec 2012.

No information is available on the current funding status for development, although the sourceforge site’s bugtracker continues to list new entries and responses. 

WCT encourages developer participation, publishing a Developers Guide with the latest release.

Platform and interoperability

WCT was written in Java and designed to run in Apache Tomcat. It has been tested on Red Hat Linux, Solaris, and (to a lesser extent) Microsoft Windows. Three relational databases are officially supported: Oracle, MySQL and PostgreSQL.

The software itself makes use of part or all of several other open-source components, including: Heritrix; Wayback; Acegi Security System; Apache Axis; Apache Commons Logging; Hibernate; Quartz; and Spring Application Framework.

Functional notes

An important functionality of the software is the ability to collect, store, and abide by harvest authorisations, i.e. permissions to download from the copyright holders. WCT also creates separate administrative levels, so that those who set up the harvests do not necessarily have the authority to have the system actively begin them.

All material is captured in ARC format; since WCT incorporates Wayback, access within the system is not a problem. However, those who collect material with the ultimate goal of archiving it in a separate system must take the format into account.

Documentation and user support

The project site includes a well written quick-start guide and user manual, although the manual includes heading sections missing the corresponding text. The site also includes a developer guide, published with release version 1.5.2. Links to the advertised wiki and FAQ sections are currently broken, forwarding to the sourceforge developer page instead.  More information about the project can be found in an informative article published in Ariadne Issue 50.

The primary forum for technical support appears to be an active “webcurator-users” mailing list. While bugs continue to be posted on the sourceforge bug/ feature request tracker, the last addressed item was in February 2011.

Usability

WCT is specifically designed to be operated by non-technical users such as librarians, with a simple and relatively intuitive GUI. Installation, however, is difficult and most likely requires tech support.

Expertise required

Setup, especially if it includes links to an archival repository, requires system administration knowledge. Users must have a comprehensive understanding of their institution’s collections policies when designing harvests.

Standards compliance

WCT allows users to add basic Dublin Core metadata to the material.

Influence and take-up

WCT is used by the National Library of New Zealand, the National Library of Norway, and the British Library. The sourceforge site lists nearly 8,300 downloads as of December 2011.

Last reviewed: 
5 June, 2013