Archive skills and tools for historians
Sean Cunningham
Not all historians choose to, or need to, use archival collections to conduct innovative and exciting research. Some historians continue to command the respect of their peers (and their publishers) at a considerable physical or intellectual distance from the unique manuscripts and papers usually associated with archival collections.
Nevertheless, many professional historians associate archival research with their rite of passage into the profession. At some point in their careers most scholars have devoted several long weeks to the systematic examination of the carefully sorted primary sources in their chosen field of study. Few historians would disagree that the refereed article, monograph or scholarly study requires a range of evidence, some of which will be extracted from documents held in archives, and used as the basis of discussion and argument.
Since the nature and format of records relative to particular subject areas and periods of history has changed little, so one might expect that the evolution of the associated archival skills of historians has been a slow process. This has probably been the case for decades, if not over centuries of historical and antiquarian study. This particular aspect of the discipline of historical study, however, is now subject to a process of great change.
Historians’ interaction with archives has been dominated by the need to identify the range of material within collections, to access relevant documents, and to interpret their contents productively. By no means does all investigation of the past require the consultation of the written word. Although diaries, letters, deeds, accounts, enrolments, depositions or notebooks might come to mind when archival collections are visualised, archives now contain film, photographs and sound collections.
These sources lend themselves to particular types of research activity limited to the recent past by nature of the widespread adoption of such technologies as archival sources. Despite the non-paper format of this material, it is still likely to be identified and accessed through printed or published guides, finding aids, indexes, lists and catalogues.
The process by which the documents were approached was traditionally explained in studies produced by archivists and administrative historians. The range and depth of this material varied, but as a body of work it was intended to provide a solid foundation upon which to build archival research skills. Thus, T. F. Tout’s Chapters in the Administrative History of Mediaeval England (1) remains the clearest guide to how departments of crown governance worked before the 15th century.
Other guides have appeared as the range of archival sources has expanded. From Sir Henry Maxwell-Lyte’s Historical Notes on the Use of the Great Seal of England,(2) to Anne Thurston’s Sources for Colonial Studies in the Public Record Office,(3) or Michael Roper’s The Records of the War Office and Related Departments, 1660–1964,(4) historians have sought a reliable context for the sources that they use. Institutions such as the Public Record Office (PRO)/National Archives (TNA) have also provided highly specialised handbooks, such as David Crook’s Records of the General Eyre,(5) that now serve as standard scholarly introductions to groups of sources or spheres of historical study.
Many archives also produce a range of more specific subject-based research guides, memoranda and source sheets. Others have complex introductory notes to specific document collections. These guides serve to steer historians through archival collections. They not only introduce series of documents and their interrelationships, but also offer some explanation of technical issues such as abbreviations, palaeography, archaic terminology and obsolete referencing systems.
This corpus of information has been routinely accessible only through the collections in major libraries or in the archives where the documents themselves are housed. The process of background research, interpretation and consultation of original documents has in the past taken place within the scholarly environment of major libraries and archival search rooms. This activity was the essential preliminary stage in the approach of manuscript sources. However, fundamental changes are now happening to the way historians access archival collections and how they interpret archival sources.
First, archival source material is changing from paper to born-digital documents, images and website content. In areas such as the preserved public records of government activity, archives are nearing the point where paper files cease to be the main materials accessioned from government departments. In future, databases, word-processed documents, archived emails, spreadsheets and digital presentations are more likely to form the archival sources upon which core aspects of national history from the late 20th century onwards must be based.
This is a fundamental fact, and an immense departure from centuries of records management based on parchment and paper collections. TNA is taking rigorous steps to address the problems associated with the indefinite storage of, and future access to, digital government records. Contemporary historians are thus obliged to make a shift in approaches to the format of their primary sources, and future historians will have to follow.
Second, digitisation of paper and parchment documents, both as part of major projects and as the routine business of archives, is altering how archival material is utilised by historians. There is no doubt that many historians are heavily encouraged by their institutions to develop and lead research projects funded by major grant-giving bodies such as the Arts and Humanities Research Council (AHRC), the Economic and Social Research Council (ESRC), the British Academy, the Joint Information Systems Committee (JISC) and the Heritage Lottery Fund (HLF).
Since many archives cannot lead on such bids because of their uncertain status as research institutions, partnership funding bids between the archival and university sectors are now common. Although this relationship has existed for decades, the rise of affordable digital technology and the opportunities of access presented by the Internet are driving new strategic relationships between historians and archivists in the formatting and accessibility of archival collections. To some extent, historians are now far more involved than before in the presentation of archival material to research audiences. Such a reduction in the distance between archival and historical research activity is very welcome, and will strengthen the basis for future collaboration between sectors.
Complex search engines, such as those developed by the Centre for Computing in the Humanities for a pioneering AHRC-funded project – the Fine Rolls of Henry III – are challenging fundamental approaches to archival research. Where sophisticated digital ontologies are mapping relationships between strands of data, the research possibilities of groups of manuscripts previously seen individually in their original format are now expanding at a bewildering rate. The successful reception of this type of hybrid catalogue/calendar/image website, which is also producing printed resources, is helping to create a standard for means of access to large volumes of data alongside images of original documents.
Digitisation of original documents by several projects has accelerated the creation of ‘virtual reading rooms’. For example, the Anglo-American legal tradition website at the University of Houston now contains over 2 million digital images of medieval and early modern English legal records from the central law courts at Westminster. These images are accessed via a no-frills website without metadata or search functionality and are arranged by the court, regnal year and law term to which they relate. In order to get the most from this resource, users still require the requisite traditional interpretative archival skills in Latin, Anglo-Norman French and palaeography, and experience of the abbreviated format of the documents created by the medieval legal system.
Without accurate reproduction of the original internal manuscript referencing structure, it also becomes harder to cite such images as if one had consulted the original. The inferred emphasis from the website owner is that the online surrogate of the document, and not its parchment original, should be cited by historians using the resource. This problem presents difficulties of its own, yet the key advance remains: for those scholars who find themselves thousands of miles from the UK National Archives at Kew, this site places the documents at their fingertips.
The Fine Rolls example above demonstrates what can be achieved when historians, archivists and technologists co-operate productively. The Anglo-American legal tradition site shows how digital technology now enables a very small number of researchers to capture vast amounts of data – in this case unadorned document images. With the right type of sustained backing from a stable institution, striking developments in access to the archival resources available to historians are now within the compass of determined individual scholars.
Technology is facilitating great leaps in the development of systems for constructing search functionality. We might be nearing the point where off-the-shelf toolkits become available for local archives and history societies to produce and maintain their own document-based websites and search databases. Since the pressure upon historians to secure financial backing for research projects means that competition for major project funding has never been fiercer, a possible route for further collaboration between historians and archivists is to grasp this affordable technology and create resources tailor-made to specific research projects or groups of sources, without the need to rely on the lottery of major grant applications.
Potential difficulties are presented by this rapid expansion in digital technology. As archives give permission for digital images of entire document series to be captured by individuals, or licensed for publication as web-based resources, it might become harder for historians to secure funding for major analytical projects if digital versions of the relevant documents already exist as part of unrelated websites.
Furthermore, although backing for the maintenance of websites and databases might seem secure at present, and might be guaranteed for the foreseeable future, there can be no absolute certainty that institutions will offer the same assurances in 100 years’ time. Regular maintenance is also necessary to make readable the format of today’s digital resources far into the future. Although the medium of print offers limited access when compared to web-based resources, it remains a format that has proved to be stable. With the end of funding for the Arts and Humanities Data Service (AHDS) in April 2008, there is no longer a central repository for the data generated by historians’ research projects.
If, as seems likely, online archival sources proliferate expansively, and remote viewing of digital document images becomes a mainstay of access to archival collections, should not some central agency co-ordinate the preservation of this digital information? Whereas AHDS stored data, there is not yet an equivalent repository for data and digital images generated by funded research projects.
There has been remarkable investment from grant giving bodies in various fields of historical research. The digital products of this support represent resources upon which the entire community of historians can draw, but at present the long-term sustainability of, and access to, these resources remains unclear. Without careful consideration and discussion between individual grant holders, universities, research councils, grant giving charities, archives and IT systems developers, datasets and digital images might become locked into obsolete software formats and rendered inaccessible as technology moves on rapidly. Historians and archivists have much work to do in order to make digital resources both sustainable and accessible in the long term.
A third broad area where historians’ archival skills must develop concerns catalogues and indexes. The arrival of unique digital records and digital surrogates of existing paper material is forcing a change in the processes through which archived documents are accessed. While developments in accessibility might allow historians to draw archival references together more productively, the main priority for archives that are producing digital images of their collections – either through licensed partnerships, or from their own resources – is to create full descriptions and comprehensive search functionality. The fuller the archival description of documents, the easier it is for historians to devote their analytical and interpretative training to unlocking the evidence that those documents contain.
The advent of born-digital records and digital surrogates has driven the development of searchable catalogues, both for documents that have been converted into digital format, and those that remain in their original form. Catalogues and indexes therefore remain the keys to accessibility. All archives are now working to improve the level of detail and contextualisation offered by the descriptions of the records that they hold.
Where online catalogue descriptions of documents are particularly full, the need to consult original papers or manuscripts diminishes. Most historians require the information contained within source material, and if that can be supplied in surrogate or comprehensively summarised form, then the need to view the physical primary source becomes less necessary (this was always the function of published calendars).
The almost incomprehensible volume of information on the Internet, and the speed with which relevant documents can be located, is also affecting the expectations of those engaged in archival research. The proliferation of online catalogues has created the mistaken assumption that all archival documents are fully described and accessible through electronic, web-based catalogues. This is not the case, since conversion and restructuring of existing lists and indexes is a complex and resource-intensive process for archives. Many decades of such activity lie ahead, even in the best resourced archives, before online catalogues come to represent a complete inventory of particular archival holdings.
This assumption does not really arise among those historians who have laboured previously with paper and manuscript lists in archival search rooms. Their skills have had to become more adaptable as the possibilities of new technologies have been embraced. For those who have entered their careers with access to online catalogues and documents as a familiar basis for their research skills, the adjustment to paper indexes found only in archives, contemporary registry systems and layered arrangements of former references can be something of a shock.
A new archival skill historians must adopt relates to the process of gaining accurate information on the completeness of online catalogues when compared to more comprehensive lists that existed in paper form. Scholars have always balanced their willingness to trawl archival collections against the level of detail contained in catalogues and indexes.
A fourth major change in the range of historians’ archival skills stems from the proliferation of activity to convert paper lists and indexes into Internet-based searchable catalogues. The most dramatic archival development of the past 15 years has been the phenomenal rate at which the Internet has permitted linked catalogues to comb through the descriptions of several collections at once. The Access to Archives (A2A) website, for example, links electronic catalogues of around 400 archives within England and Wales. The National Archives’ Global Search facility links catalogues, databases and project websites in a seamless search. Navigating these websites under a single keyword produces in seconds what, only ten years ago, would have taken months to achieve by page-turning and physical travelling between institutions.
Since no standard has yet been adopted for the structure and presentation of digital image collections, some difficulties are emerging as systems are developed for specific, rather than universal, audiences. In the recent past, publishers’ editorial conventions and the personal preferences of editors had the effect of screening certain kinds of evidence and information from printed calendars and guides. Such editions were the pre-digital equivalent of the websites and databases that are now revolutionising broad access to archival information. There are now numerous examples where search engines developed primarily for commercial and mass-market appeal often do not serve the academic historian fully.
For example, the data involved in indexing the online versions of the 1841–1901 census records, runs to tens of millions of names for each survey. Yet the search options are constructed around genealogical investigation. Historians hoping to use the English and Welsh census to facilitate possible scholarly studies of people engaged in particular trades in a town, migration, social mobility or surname distribution, will have to work hard to extract data from resources constructed for single name searching.
Many researchers seeking digital copies of documents contained in TNA’s DocumentsOnline website are also steered towards searches for individuals (although the range of core executive documents is expanding rapidly, and more flexible searches are possible with some imaginative manipulation of the search fields). Clearly, the need to generate income makes this genealogical approach an economically sustainable option. It does, however, limit the broadest utility of some digitised documents, and this trend has not benefited academic historians as well as it might have done.
Several catalogues have employed inconsistent standards to the entry of keyword data upon which searches are based. The shortcomings of the TNA’s Catalogue in this area are perhaps more prominent because of the sheer size of its database, and the fact that it is the product of several years’ worth of layered development and overlapping data entry. Searching for early wills on the DocumentsOnline site, for example, frequently requires practical knowledge of the variants of medieval and early modern spellings of personal names, since searches will only return results if the search terms match exactly the catalogued content (or its stem). This is also the case in other catalogues, where results might be missed unless alternate spellings of search terms are considered.
The skill levels historians need in order to interpret and analyse manuscripts have remained consistent throughout these developments in the format though which documents are studied. Digital catalogues have focused primarily on problems of access to manuscript collections; especially where previous arrangements have been explained as part of the process.
The skills required to interpret and analyse the contents of web versions of documents remains less well developed, since generic help is difficult to present in anything but a basic level within online resources. The onus remains with historians to acquire and extend the skills they need to extract the required evidence from relevant documents. For those historians who already possessed mastery of their documents, the digital world of access to, and presentation of, primary source material has made few new demands on their fundamental skills in administrative history, languages and diplomatic, etc.
Archivists might be forgiven for thinking that in some cases historians’ archival skills are now focused primarily upon mastering the technologies of their laptop and digital camera software in order to minimise the time they spend within search rooms. The process of understanding the content and context of the documents under investigation is becoming something that historians no longer have to attempt within archives to the same extent as they have done in the past. Although a gross generalisation, digital technology is allowing this phase of the research process to be conducted in front of a computer rather than before a manuscript or departmental file. The ability to capture hundreds of digital images in a day (in those archives that permit free use of digital cameras) has certainly altered the focus of archival researchers.
The massive transformation underway in access to archival information should not cloud the fact that the discipline of history still requires the full range of analytical and interpretative skills that have always been at the core of the best type of archival research. The format through which primary source material is accessed is certainly changing: from paper or parchment indexes and documents studied within archival search rooms, to searchable catalogues and digital images of documents viewed by computer. The content of these texts and documents remains the same, so in one sense the interpretative archival skills necessary to decipher them are unchanged. Historians’ archival energies are now directed wholeheartedly at sifting through the proliferating range of online information relevant to their field of study.
There is no doubt that the best Internet resources for historical investigation are revolutionising access to texts and documents and developing the way in which archives are used. But is there not already a danger that historians’ reliance upon digital resources is mismatched to the completeness and dependability that those resources presently offer? With most historians physically distant from major archival collections, it is easy to understand why comprehensive and accurate resources such as British History Online (BHO) have become so well used and respected within certain research communities but not all historians are so well served.
The discipline of archival research is therefore in a period of flux. The first stages of contextual research can now be carried out online. Calendars, textual summaries of documents, articles and administrative histories are accessible via websites like BHO, Google Books or JSTOR. Consultation of major collections of digital document surrogates can be conducted from any suitably equipped PC or laptop. The physical separateness of historians from archives is progressing as archival information and parts of archival collections become almost instantly accessible online. An entire range of new archival skills related to information manipulation and access are being added to traditional techniques of document interpretation.
The benefits brought by immediate universal access to sources via the Internet are raising new issues over completeness, reliability and sustainability of archival resources. Historians and archivists are at the centre of these developments, and will continue to work together closely to ensure that the discipline of history continues to benefit from the recent rise of digital online resources.
- T. F. Tout, Chapters in the Administrative History of Mediaeval England (Manchester, 1933).
- Henry Maxwell-Lyte, Historical Notes on the Use of the Great Seal of England (London, 1926).
- Anne Thurston, Sources for Colonial Studies in the Public Record Office (London, 1995–8).
- Michael Roper, The Records of the War Office and Related Departments, 1660–1964 (London, 1998).
- David Crook, Records of the General Eyre (London, 1982).
Dr Sean Cunningham is Medieval and Early Modern Records Manager at The National Archives
