Partners: Elizabeth Bedford, Richard Boyechko, Jose Cols
SSEC Engineers: Niki Burggraf, Vedant Somani
Background
University of Washington Libraries is creating an automated workflow to collect publications from UW researchers to centralize and preserve their work and make it openly available to the world in UW’s institutional repository, ResearchWorks. Research is published across a wide range of venues and repositories, and institutions benefit from maintaining a comprehensive, stable record of their own affiliated scholarship — both as a matter of institutional identity and as a hedge against the inevitable changes that affect any external repository over time. By gathering a collection for the campus community, the project aims to increase the accessibility and longevity of UW-published research and provide a potential model for other institutions.
Software Problem
The close to 400,000 scholarly articles authored by UW-affiliated researchers are distributed across federal repositories such as PubMed Central and ERIC, spanning a diversity of venues, disciplines, and publication styles. This breadth requires a system with robust architecture capable of both harvesting from these sources and storing the articles and their metadata in a well-preserved manner. UW Libraries sought to automate the aggregation of this distributed corpus of UW researcher work into a single institutional home for use by students, staff, faculty, and the public. Beyond collection, the system must also classify each article by rights status — identifying works covered by UW’s faculty open access policy, works published under Creative Commons licenses, and works requiring further review — so that materials can be made responsibly and legally available. The software needed to be modular, customizable, and well documented to ensure ease of future use by other institutions with similar use cases.
Software Solution
SSEC collaborated with UW Libraries in the initial phases of this project, setting up the GitHub repository in the uw-ssec GitHub organization, building in pixi as a package and environment manager, and reviewing pull requests. SSEC guided the system design process, helping establish database schema and architecture capable of supporting a large amount of data. The Open-IRE team then built solutions to address disambiguation, scrape essential metadata such as licensing information, DOI, and existing access constraints. Thanks to careful modular design and standards development the system can easily expand to other sources without needing to redesign the database schema.
Impact
With close to 50,000 articles already collected, the Open-IRE team is working to process additional materials, resolve rights questions, and accommodate further standards. As a direct result of this system, UW Libraries will be able to easily provide a stable, institutionally maintained home for past and future UW scholarship. Moreover, through rigorous documentation, it can be adapted by other institutions looking to provide similar open access infrastructure for their campuses and communities.