Scriblio MATC Project Final Report

[innerindex]Scriblio, formerly WPopac, was initially developed for internal use at Plymouth State University. It was the Andrew W. Mellon Foundation’s recognition of the project in 2006 that truly raised our attention to the interest that the larger library community had in the project. Though we had nominally released it under the terms of the GPL and many aspects of it were discussed in detail in Casey Bisson’s blog, the Mellon award also forced us to “productize” the software in a way that would allow others to easily install and use it. Among the small group with whom early news of the award was shared, many expressed hope that Scriblio could be made to serve the needs of small, rural libraries of the type New Hampshire is rich with. As community outreach is a cornerstone of Plymouth’s mission, we have adopted a strategy to make Scriblio serve the needs of a broad spectrum of users that begins with rural public libraries and extends up to the needs of large academic institutions.

The challenges to this are vast. Public and academic libraries often use very different systems, and an average public library serving a community of 2,500 people subsists on an annual budget of about $35,000, while academic libraries enjoy significantly larger per-patron budgets. Yet all libraries are struggling with the challenges of delivering relevant services in the 21st century and the significant changes this requires. Why can’t we make Scriblio easy enough to use and manage that it could serve the needs of these users?

In late 2004 Bisson identified three essential problems with library software: poor usability, findability, and remixability. Today, Scriblio answers each of those with ease. Like most “next generation” library systems, Scriblio solves the usability challenge with improved keyword indexing and faceted search and browsing features. But Scriblio distinguishes itself in its focus on making its data findable in internet search engines like Google and in making it remixable so that others can develop innovative services and mashups based on library data. Further, we are actively working to identify and address other problems that libraries face. One unique feature of Scriblio is the way it co-mingles the library catalog with web content of the library, making it easier for the patron who arrives at a library website with a question to answer it without first having to struggle with the issue of where to ask it. Interested in knowing if your library offers wireless internet access? Just search and you will find. At Cook Memorial Library in Tamworth NH, the top result is an article explaining how to get online, followed by some books on how to setup your own home network.

Scriblio has proven itself a flexible and capable platform for the development and deployment of innovative library services, while also remaining easy to use and implement.

The Data Problem

The technical challenges to broad implementation of Scriblio or similar next generation library tools are quickly being overcome, but a large number of non-technical issues are revealing themselves as potential barriers. Two examples: the availability of quality data, and cataloging standards that are not well suited to faceted browsing.

The conflicts between cataloging standards and faceted search and browse tools was the subject of a pair of presentations given by Bisson at the ALA Midwinter and Annual meetings in 2007. The problem is that a subject heading such as “art, Asian” does not work as well in faceted search systems as two separate headings for “art” and “Asian”. The cataloging community appears understanding of the problem and interested in solving it, but applying such a sweeping change on such the large number of subject headings is an enormous political, logistical, and technical challenge. What is needed is a mechanism to rapidly evolve cataloging standards to suit the needs of faceted tools as well as a means to share the data enhancements that will result from those evolved standards.

Another issue is that of access to data. Publishers now distribute cataloging records for their published works for free, and the back catalog of records representing most previously published work can be had at relatively low cost. What doesn’t exist is an easy, inexpensive mechanism to share that data among libraries. This issue is particularly important to small libraries, and was identified in our project proposal to the Foundation.

A public, open source licensed data repository offers a solution to both of those problems, and Scriblio is pursuing two related strategies to deliver such a solution. We are building a repository based on Library of Congress records and a set of Application Programming Interfaces (APIs) that allow enhancement of a library’s local records with data from that repository. An early version of that API is already a valuable part of the Scriblio software, and we are continuing our efforts to build features and extend the collection. Additionally, we are participating in the Internet Archive’s efforts to build Open Library, a larger-scope collection of records with wiki-like editing features and links to full-text where available. Taken together, we believe these two approaches will deliver real value to libraries by lowering the cost and complexity of getting and sharing data and improving the quality of the data available.

The Success of Collaboration with WordPress

The founding philosophy of the Scriblio project is simple: use common tools to solve unique problems. Scriblio has grown significantly since then, but that philosophy continues to drive our development efforts whenever we encounter a new challenge or identify a new problem to solve. The first, and most obvious expression of this was in choosing to build Scriblio on top of WordPress. What, after all, is a library catalog other than a database driven website? And what more tested or extensible a tool for building a database driven website than WordPress?

Today, WordPress powers approximately three million sites, two million hosted at WordPress.com alone. That number of users, in addition to the vibrant community of developers building themes and plugins, makes WordPress one of the best understood and most supportable web applications available. Furthermore, the relationship between the open source WordPress community and commercial participants, including Automattic, the commercial entity that operates WordPress.com, has proven itself to deliver real benefits to all.

For Scriblio, building on top of the framework that WordPress provides has delivered a number of gifts. The commenting system, now recognized as an essential component of any library system, has many developers hardening it against attack or spamming, while the challenge of pleasing a large number of non-technical blog authors keeps those developers focused on making those security features easy, even fun to use. Edward Spodick, of Hong Kong’s University of Science and Technology, pointed to another benefit of the relationship with WordPress in a February 10th 2008 email to the Web4Lib list. Speaking on the topic of how libraries are represented in common internet search engines, he explained:

…many ILS systems, including both the III and VuFind implementations, suffer from using a generic TITLE tag in the HEAD of the html – to the title tag, which is used for displaying Google search results, will just be something like the following for every record:

     "Hong Kong University of Science and Technology"
     "Library Resource Finder: Record Holdings"

Not very useful when what the user would want would probably be the title of the item.
The Scriblio implementation does a better job on this aspect, at least in our implementation, with things like:

     "HKUST Library Catalog — Japanese popular music : culture..."

as http://catalog.ust.hk/catalog/archives/710731

But Scriblio does not deserve credit for this. It is the large WordPress community, which includes a number of “white hat” search engine optimization experts, that long ago realized how important it is to put the title of the post in the HTML title that deserves credit.

And the Scriblio project has enjoyed opportunities to contribute to the WordPress community as well. Though Scriblio has learned much about optimization of high volume sites from the system administrators of WordPress.com, there are few blogs that have nearly as many posts as an average library has books. Because of this, we have identified and fixed a number of performance bottlenecks in WordPress that are only evident in such large databases. One recent example is Ticket #5649, where a change proposed by Scriblio was committed to the baseline code within an hour of its submission. 

The Challenges of Collaboration with WordPress

Shortly after the Mellon Foundation announced the award to the Scriblio project, the WordPress core developers reversed their longstanding position on tags and announced that the next release would include tag support. This is significant because metadata such as author or subject is functionally equivalent to tags in Scriblio, and much of the Scriblio code was devoted to managing those tags.

That announcement, combined with significant uncertainty about how tags would be implemented stalled development on those aspects of Scriblio. The matter was further exacerbated by delays in developing the promised tag components in WordPress. Concern about how the Scriblio community would manage this change led to delays in broadly releasing the Scriblio code (the significance of the change is evidenced by the fact that, as of this writing, only one of the three sites to go live prior to it have since successfully upgraded to the current code). In retrospect, the decision to delay that public release was regrettable, and the challenges of Scriblio’s close dependancy on WordPress were larger than at first anticipated.  

However, we do not foresee any changes to WordPress that will affect Scriblio on a similar scale. And we expect that as Scriblio matures it will be better able to endure or participate in such changes to the WordPress roadmap in the future. The benefits of such collaboration are clear: from usability to security to search engine optimization and more, Scriblio directly benefits from the attention the WordPress community gives to every detail.

Scriblio As Community Builder

Much of New Hampshire defies our stereotypes of a digitally connected community. New England town meetings and our cherished main streets would seem to defy the notion of a digital community, yet two early adopters of Scriblio have learned otherwise. Over the past decade, Berlin New Hampshire has seen the closure of all its paper mills, the primary industry and employer in this town of about 10,000 people near the northern tip of the state. The median household income and home values are about half the New Hampshire average, and the loss of jobs has forced many residents to leave, making it one of the few towns in the state with negative population growth.

The town may be struggling, but a sense of civic pride and optimism is evident among those who remain. The comments at Beyond Brown Paper, a Scriblio-powered site that offers the photo archives of the Brown Company up for searching, browsing, and most importantly commenting, reflect the proud history of the town’s residents. The photo collection includes approximately 15,000 items from the late nineteenth century to about 1965. And though most of the photos include notes describing the date and location, the history the collection represents is best told by former workers and their family in the comments. A series of comments on one photo name one of the people picture in the photo and describe the context and industrial process surrounding it. Near the end of the list of comments appears one from the nephew of the named person confirming that the pictured person is indeed his uncle. And those are just a few of the over 200 comments on the site that help bring the photos alive and reveal that the archive itself is only a collection of artifacts, the real history lives within the residents and former employees. The aim of Beyond Brown Paper is to collect that social history, but its success may be something greater. A group of local high school students used it to prepare a large-scale local history project, and anecdotal evidence exists to suggest that the site is encouraging increased use and awareness of technology. 

Like many communities in New Hampshire, residents of Tamworth have only limited access to broadband internet. The small, close knit town of 2,500 has a 30,000 volume library that is open four days per week. And yet within months of introducing their Scriblio site the comments hosted a discussion about sustainable agriculture that led to further in-person discussions on the subject. The discussion wasn’t prompted, it erupted organically when a resident spoke of her reaction to Michael Pollan’s Ominvore’s Dilemma in the comments and others joined her.

The most touching story of how Scriblio builds community, however, is much simpler. The Cook Memorial Library director and her two staff believed they knew pretty much everybody in town, but Scriblo introduced them to a resident they hadn’t met before: a homebound paraplegic with a passion for reading, and it all started with a single comment. Scriblio created a new opportunity for this patron to participate in the community that had not before been possible, and the library is more inclusive and welcoming because of it.

Usability

To ensure that Scriblio development is meeting the needs of the majority of library users who often have little or no experience with libraries, Plymouth approached usability consultant Robert Hoekman, Jr., to perform an expert usability review of the software. Lamson Library’s implementation was selected as the object of the review because the library offers a rich set of supporting online services and the site is running the most current code. Though the Lamson site looks very different from an out of the box implementation of Scriblio, the way the system works and the way the data is presented to the user is the same. Certain aspects of the Lamson site, such as the live reference chat, are dependent on individual libraries, but we offer the implementation and integration of those features as demonstrated at the Lamson site as an example of what is achievable in Scriblio implementations elsewhere.

Hoekman is the founder of Miskeeto, a product development and web design consultancy, and author of Designing the Obvious and the forthcoming Designing the Moment, both from New Riders Press. 

In his review, Hoekman has identified details that may need improvement, but he also offered overall praise for the project:

Scriblio succeeds in a number of areas where current alternatives fail. It is not only one of the best library systems on the web, it is one of the better e-commerce and knowledgebase systems. […]
OPAC developers everywhere should study Scriblio, learn from it, and start meeting the bar that Scriblio has set. 

Outreach

More than 25% of America’s 9,207 public libraries serve communities of smaller than 2,500 (NCES Public Libraries in the United States, 2004) with an average annual budget of about $35,000 (Rural Sustainability Workshop, 2004). All told, about 60% of US public libraries serve communities of fewer than 10,000 people (NCES Public Libraries in the United States, 2004). Those numbers and Plymouth State University’s commitment to community development in our rural state directed our efforts at ensuring the advanced features offered by Scriblio were available to those libraries. Therefore, we began our outreach activities by identifying a representative rural public library as a development partner. 

In a project led by Lichen Rancourt, Head of Technology at Manchester City Library, Manchester NH, we identified Cook Memorial Library in Tamworth New Hampshire. The library director and board committed to the partnership in late 2006 and went live with a Scriblio beta site in early 2007. Prior to the launch of their Scriblio beta site, the library had a limited online presence and no online catalog; further, their technical capabilities and experience with current web technologies was limited. Though they were eager to participate and to deliver enhanced services to their users, they had to overcome a number of challenges in doing so. The library staff made a commitment to personal skill development, but their experience with Scriblio also led to significant improvements in the ease of installation and management of the application. That Scriblio can now be installed and configured in about ten minutes is a direct result of lessons learned from that partnership.

About.Scriblio.net is the official site for news, information, downloads, and documentation related to Scriblio, the Subversion repository is hosted at svn.Scriblio.net, and we have a Scriblio email group hosted by Google Groups. Postings at those sites, as well as at the personal blogs of members of the project have formed an important aspect of our outreach efforts. The efforts of Plymouth’s public relations office have also led to coverage of Scriblio in The Chronicle of Higher Education and by New Hampshire Public Radio.

We have also done a number of presentations and media appearances as part of our outreach efforts, including:

• Top Tech Trends, Ontario Library Association Superconference, February 2 2008 (Bisson)
• Scriblio, Ontario Library Association Superconference, February 1 2008 (Bisson)
• Libraries on the Social Internet, Rhode Island Office of Library and Information Services program, December 6 2007 (Bisson)
• Keynote, IT-Faggruupen meeting on Today’s Hottest IT Trends (Copenhagen, Denmark), November 16 2007 (Bisson)
• Keynote, New Hampshire Library Association Annual Meeting, November 1 2007 (Bisson)
• OPAC for Web 2.0, Internet Librarian, October 31 2007 (Bisson)
• Young Librarians, Talkin’ ‘Bout Their Generation, interview in The Chronicle of Higher Education, October 19 2007 (Bisson and others)
• Scriblio and Tamworth Library, New England Library Association annual conference, October 16 2007 (Rancourt)
• Web 2.so, Tamworth Public Library event, September 26 2007 (Rancourt)
• Web 2.so, New Hampshire Library Association workshop, September 24 2007 (Rancourt)
• Bringing The Library To The User, AALL Annual, July 15 2007 (Bisson)
• Transforming Your Library, ALA Annual, June 23 2007 (Bisson)
• Web 2.so, ALA Annual (LITA Social Software Showcase online-only presentation), June 23 2007 (Rancourt)
• Small Steps, ALA Annual (LITA Social Software Showcase online-only presentation), June 23 2007 (Bisson)
• Faceted Searching and Browsing in Scriblio, ALA Annual, June 22 2007 (Bisson)
• Scriblio discussion at Simmons College, April 18 2007 (Rancourt and Bisson)
• Keynote, Boston Library Consortium Annual Meeting, April 12 2007 (Bisson)
• Libraries in the Digital Age, New Hamshire Public Radio interview on The Front Porch, March 1 2007 (Rancourt and Bisson)
• Libraries vs. The Google Economy, ALCTS Forum on the Future of Cataloging at ALA Midwinter, January 21 2007 (Bisson)
• Metadata and Faceted Searching, CCS Cataloging Norms Discussion Group at ALA Midwinter, January 20 2007 (Bisson)

Implementations

Because of the previously mentioned changes to WordPress’ support for tags, the Scriblio team did not actively promote implementations until version 2.3 (released fall 2007). This greatly delayed adoption of the software, but a number of sites are already online: 

• Beyond Brown Paper launched in December 2006 and represents the archive of photographs from the former Brown Company, a manufacturer and formerly the major employer in communities of northern New Hampshire. Scanning is ongoing, but the online collection already includes over 12,000 objects and more than 250 comments.
• Cook Memorial Library, Tamworth NH launched in February 2007. The library had no online catalog and only a limited online presence prior to implementing Scriblio, but the small staff and community of 2,500 have embraced it. The collection of about 20,000 items is matched by a fast growing body of comments and blog posts that are bringing the community online.
• Lamson Library, Plymouth State University, launched a completely revamped site based on Scriblio in summer 2007. The site represents some of the most in depth thinking about how to offer library services online, what content and interactions are important to users, and the shape that library systems must take to successfully deliver those services and information. In addition to over 300,000 records, the site includes almost over 200 pages of librarian generated content and a large number of integrated services. Usage has grown steadily since its release and now averages about 60,000 page loads per day, significantly greater than the 2,000 daily page loads the library’s previous catalog served.
• Boston University School of Theology’s History of Missiology collection, launched after several years of effort at the 2008 Costas Consultation in Global Mission conference on February 8 2008.
• Hong Kong University of Science and Technology‘s Ki Tat LAM, Head of Library Systems, announced the university’s adoption of Scriblio in a February 1 email to the Scriblio group: “HKUST Library has built its Next-Generation Library Catalog based on Scriblio…. It currently has more than 710,000 bibliographic records imported from our INNOPAC/Millennium catalog. We have also done some customization to make the CJK (Chinese, Japanese, Korean) importing and searching work.”

And one especially notable example of how revolutionary Scriblio really is: Robin Hastings, IT Manager at Missouri River Regional Library, installed Scriblio just to see how it worked. She reported her success in her blog:

…even with a non-standard set up (oh, what I would give for a single LAMP server…) I was able to take the instructions, install the files, do some bug-hunting, fix those pesky bugs (mostly) and have a working (and workable) version of Scriblio on my server in about 2 hours. I even have a comment!

Hastings’ test installation hasn’t (yet) led to adoption of Scriblio by her library, but it shows how software that’s easy to use is also easy to experiment with. It is very difficult to know how many people have downloaded Scriblio (the SVN server doesn’t keep such statistics), and many stories like Hastings’ come from people who’ve not joined the mail list, but the interest from others in the library community and our expectations for future grown are high.

Budget

The Foundation generously awarded the project $50,000 in December 2006 and the funds were invested according to the instructions of the Foundation.

Purchase of the Library of Congress’ catalog (used to support record enhancements) consumed the largest portion of the funds, but the anticipated purchase of servers for the project was obviated by a gift of servers from Dell Inc. The funds that had been budgeted for the servers were redirected to support outreach efforts, especially travel to conferences. A few contract staff and consultants joined the project to help improve the ease of use of the administrative and management aspects of the software and it’s ease of installation.

Foundation support allowed Project members to attend ALA Midwinter (Rancourt and Bisson), ALA Annual (Bisson), and Internet Librarian (Bisson), and offered significant opportunities to represent the project to the library community as well as collaboration. Bisson’s participation in WordCamp, the WordPress developer’s conference, was also supported with Award funds.

Key people:

• Casey Bisson, Information Architect, Plymouth State University. Bisson began work on what became Scriblio as he attempted to answer a question that emerged from Lamson Library’s search statistics: why are so many students searching for “Sociology of Education” and not finding the LC subject heading for it, “Educational Sociology.” Bisson’s work on the project is an official part of his duties at Plymouth.
• Lichen Rancourt, Head of Technology, Manchester City Library (Manchester NH), is participating in the Scriblio project outside her official duties at her library. Rancourt has long advocated for the use of Scriblio in rural libraries and led the implementation at Cook Memorial Library, Tamworth NH. Rancourt’s work related to ease of use issues in the installation and management of Scriblio was paid for with Award funds.
• Jessamyn West, librarian, MetaFilter community manager, and rural library tech guru assisted with documentation and in development of comment and privacy policies used at Scriblio.net and recommended to other libraries. West’s work was paid for with Award funds.

Additional individuals whose work on Scriblio was paid for with Award funds:

• Jon Link, founder of Atomic Lemur, a web design consultancy, developed the default theme for use with Scriblio.
• Matthew Batchelder, founder of Borkweb, a web design consultancy, developed the Scriblio.net theme.
• Randall Hoyt, Associate Professor of Communication Design at the University of Connecticut, designed the Scriblio logo.
• Robert Hoekman, Jr., founder of Miskeeto, performed the usability review.

Plymouth State University management oversight of the project was by David Berona, Director of Libraries; Christopher Williams, Director of Public Relations; Dwight FIscher, Chief Information Officer; and Susan Amburg, Director of Sponsored Programs.

License

In accordance with the terms of the Award and the interests of both the Foundation and University, Scriblio is licensed under the GNU Public License v.2, the same license WordPress uses.

Future

Open source software may be good for the community, but it succeeds because it solves problems for those who use it. For Plymouth, this is an easy question: compared to commercial offerings now available, Scriblio can be said to have saved the University hundreds of thousands of dollars in acquisition, license and support costs. Further, the staff time necessary to develop and support Scriblio for Plymouth’s use is similar to that necessary to support those commercial alternatives. Because ongoing development is limited to the library-specific features not provided by WordPress, the investment required to maintain the software is expected to remain low and Plymouth is likely to continue using and supporting Scriblio as long as it continues to deliver value and solve problems.
 
This does not blind us, however, to the challenges of developing the large and diverse community that is necessary for Scriblio to be a successful open source project. Nor does it dull our interest in making Scriblio ever more useful and accessible to a wide variety of libraries. 

Some features, such as development of a hosted solution based on WordPress MU suitable for representing consortia, OAI input and output (including eXtensible Catalog project-specific OAI features), support for additional ILSs, and OpenSearch (and Z39.50) input and output are outside the strict scope of Plymouth’s needs, but would greatly aid adoption of the software and build the community. Softer features, such as the development of reusable sample content and more discussion of best practices in online library services, would also greatly aid the project. Because a rich and active Scriblio community will lower the development costs for all participants, Plymouth is seeking opportunities to begin development on those features and expand the community.

We look forward to welcoming many new members of the Scrliblio community, to what Scriblio will become, and to the lessons and value it will return to the community and library users.

Related items

• Scriblio 2.7 released
• Scriblio en Español
• Scriblio Usability Review
• What is RSS?
• Scriblio 2.3 v4 Released

---