Reverse detail from Kakelbont MS 1, a fifteenth-century French Psalter. This image is in the public domain. Daniel Paul O'Donnell

Forward to Navigation

If I were “You”: How Academics Can Stop Worrying and Learn to Love “the Encyclopedia that Anyone Can Edit”

Posted: Feb 02, 2007 22:02;
Last Modified: May 23, 2012 19:05

---

Original Publication Information: Forthcoming in Heroic Age (2007). http://www.heroicage.org/.

Time Magazine and the Participatory Web

So now it is official: Time magazine thinks the Wikipedia is here to stay.

In its December 2006 issue, Time named “You” as its “Person of the Year” (Grossman 2006). But it didn’t really mean “you“—either the pronoun or the person reading this article. It meant “us“—members of the participatory web, the “Web 2.0,” the community behind YouTube, FaceBook, MySpace, WordPress,… and of course the Wikipedia.

In its citation, Time praised its person of the year “for seizing the reins of the global media, for founding and framing the new digital democracy, for working for nothing and beating the pros at their own game.” It suggested that the new web represented

an opportunity to build a new kind of international understanding, not politician to politician, great man to great man, but citizen to citizen, person to person.

Actually, as this suggests, Time didn’t really mean “us” either. At least not if by “us” we mean the professional scholars, journalists, authors, and television producers (that is to say the “pros”) who used to have more-or-less sole responsibility for producing the content “you” (that is to say students, readers, and audiences) consumed. In fact, as the citation makes clear, Time actually sees the new web as being really a case of “you” against “us“—a rebellion of the amateurs that has come at the expense of the traditional experts:

It’s a story about community and collaboration on a scale never seen before. It’s about the cosmic compendium of knowledge Wikipedia and the million-channel people’s network YouTube and the online metropolis MySpace. It’s about the many wresting power from the few and helping one another for nothing and how that will not only change the world, but also change the way the world changes.

Academic Resistance

This sense that the participatory web represents a storming of the informational Bastille is shared by many scholars in our dealings with the representative that most closely touches on our professional lives—the Wikipedia, “the encyclopedia that anyone can edit”. University instructors (and even whole departments) commonly forbid students from citing the Wikipedia in their work (Fung 2007). Praising it on an academic listserv is still a reliable way of provoking a fight. Wikipedia founder Jimmy Wales’s suggestion that college students should not cite encyclopaedias, including his own, as a source in their work is gleefully misrepresented in academic trade magazines and blogs (e.g. Wired Campus 2006).

And none of this is having any effect. Eighteen months ago, I had yet to see a citation from the Wikipedia in a student’s essay. This past term, it was rare to find a paper that did not cite it and several of my students asked for permission to research and write new entries for the Wikipedia instead of submitting traditional papers. Other elements of the participatory web mentioned by Time are proving equally successful: politicians, car companies, and Hollywood types now regularly publish material on YouTube or MySpace alongside or in preference to traditional media channels. This past summer, the story of LonelyGirl15 and her doomed relationship to DanielBeast on YouTube became what might be described as the first “hit series” to emerge from the new medium: it attracted millions of viewers on-line, was discussed in major newspapers, and, after it was revealed to be a “hoax” (it was scripted and produced using professional writers, actors, and technicians), its “star” made the requisite appearance on Jay Leno’s Tonight show (see LonelyGirl15).

Why the Participatory Web Works

The participatory web is growing so quickly in popularity because it is proving to be a remarkably robust model. Experiments with the Wikipedia have shown that deliberately planted false information can be corrected within hours (Read 2006). A widely cited comparison of select articles in the Wikipedia and the Encyclopaedia Britannica by the journal Nature showed that the Wikipedia was far more accurate than many had suspected: in the forty-two articles surveyed, the Wikipedia was found to have an average of four mistakes per article to Britannica’s three (Giles 2006). In fact even just Googling web pages can produce surprisingly useful research results—a recent study showed that diagnoses of difficult illness built by entering information about the symptoms into the search engine Google were accurate 60% of the time (Tang and Hwee Kwoon Ng 2006). In some circumstances, the participatory web actually may prove to be more useful than older methods of professional content creation and dissemination: an article in the Washington Post recently discussed how the United States intelligence community is attempting to use blogs and wikis to improve the speed and quality of information reported to analysts, agents, and decision-makers (Ahrens 2006).

Why Don’t We Like It

Given this popularity and evidence of effectiveness both as a channel of distribution and a source of reasonably accurate and self-correcting information, the academic community’s opposition to the Wikipedia may come at first as something of a surprise. What is it that makes “us” so dislike “you”?

One answer is that the Wikipedia and other manifestations of the participatory web do not fit very well with contemporary academic models for quality control and professional advancement. Professional academics today expect quality scholarship to be peer-reviewed and contain a clear account of intellectual responsibility. Authorship attributions are commonly found with forms of intellectual labour, such as book reviews and encyclopaedia entries, that were published without attribution as little as fifty years ago. Some scholarly journals are naming referees who recommend acceptance; readers for journals that have traditionally used anonymous reviews are frequently asking for their names to be revealed.

This emphasis on review and responsibility has obvious advantages. While peer-review is far from a perfect system—there have been enough hoaxes and frauds across the disciplines in the last twenty years to demonstrate its fallibility—it is surely better than self-publication: I imagine most scholars benefit most of the time from the comments of their readers. In my experience, the interest of good acquisition and copy-editor invariably improves the quality of a final draft.

Moreover, peer-review and clear attribution have an important role in the academic economy: they are the main (and usually only) currency with which researchers are paid by the presses and journals that publish them. In the professional academe, our worth as scholars depends very much on where our work appears. A long article in a top journal or a monograph published at a major University press is evidence that our research is regarded highly. Articles in lesser journals, or lesser forms of dissemination such as book reviews, conference papers, and encyclopaedia entries published under our names are less important but can still be used as evidence of on-going professional activity (see, for example, Department of English, University of Colorado [2007]). While it is not quite money in the bank, this transference of prestige and recognition is an important element in most universities’ systems for determining rank and pay.

An article in the Wikipedia is not going to get anybody tenure. Because they are written collectively and published anonymously, Wikipedia articles do not highlight the specific intellectual contributions of individual contributors—although, in contrast to medical and scientific journals with their perennial problem of “co-authors” who lend names to articles without actually contributing any research (for a discussion of one example, see Bails 2006), it is possible to trace specific intellectual responsibility for all contributions to any entry in the Wikipedia using the history and compare features. And while the Wikipedia does have a formal certification process—articles can be submitted for “peer-review” and selected for “feature” status—this process is optional and not very selective: authors or readers nominate articles for peer-review and certification after they have already been published to the web and the reviewing body consists of simply those interested users who happen to notice that an article has been put forward for review and are willing to comment on the relevant discussion page (see Wikipedia: Peer Review). While this body might include respected experts in the field, it also certainly includes amateurs whose main interest is the Wikipedia itself. It also, almost equally certainly, includes people whose knowledge of the topic in question is ill-informed or belongs to the lunatic fringe.

Why We Can’t Do Anything About It

Given these objections, it is not surprising that some of us in the professional academic community are trying to come up with some alternatives—sites that combine desirable aspects of the Wikipedia model (such as its openness to amateur participation) with other elements (such a expert-review and editorial control) taken from the world of the professional academy. One new project that attempts to do this is the Citizendium, a project which, beginning as a fork (i.e. branch) of the original Wikipedia, intends to bring it under more strict editorial control: in this project, “Editors“—contributors with advanced degrees—are to be recruited to serve as area experts and help resolve disputes among contributors while “Constables“—“a set of persons of mature judgment“—will be “specially empowered to enforce rules,… up to and including the ejection of participants from the project” (Citizendium 2006). Other, though far more specialised, attempts to merge the openness of wiki-based software with more strict editorial control and peer-review are also increasingly being proposed by scholarly projects and commercial scholarly publishers.

Few if any of these projects are likely to succeed all that well. While the addition of formal editorial control and an expert-based certification system brings their organisation more closely into line with traditional academic expectations, the economics remain suspect. On the one hand, such projects will find it difficult to generate enough prestige from their peer-review process to compete for the best efforts of professional scholars with more traditional, invitation-only, encyclopaedias such as the Britannica or collections published by the prestigious academic presses. On the other hand, they are also unlikely to be able to match the speed and breadth of content-development found at more free-wheeling, community-developed projects of the participatory web.

In fact, the Wikipedia itself is the successful offshoot of a failed project of exactly this sort. The ancestor of the Wikipedia was the Nupedia, an on-line, open-source (though non-wiki) project whose goal was to develop an on-line, peer-reviewed and professionally written encyclopaedia (see History of Wikipedia, Nupedia, Wikipedia, and Sanger 2005). The editorial board was subject to strict review and most participants were expected to have a Ph.D. or equivalent. The review process involved seven steps: five analogous to those found traditional academic publishing (assigning to an editor, finding a reader, submitting for review, copy-editing, and final pre-publication approval) and two borrowed from the world of open source software (a public call for reviews, and a public round of copy-editing). Begun in March 2000, the project ultimately collapsed in September 2003 due to a lack of participation, slow time-to-publication, and conflicts between professional contributors and editors and members of the public in the open review and copy-editing parts of the review process. In its relatively brief existence, the project managed to approve only twenty-four peer-reviewed articles for publication. At its suspension, seventy-four were still in various stages of review. After the project as a whole was suspended, the successful articles were rolled into the Wikipedia. Relatively few can be found in their original form today.

The Wikipedia was originally established as a support structure for the Nupedia’s open processes—as a place where participants in the larger project could collaborate in the creation of material for the “official” project and contribute to their review and copy-editing. The wiki-based project was proposed on the Nupedia’s mailing list on January 2, 2001 and rejected almost immediately by participants for much the same reasons it is frowned upon by professional academics today. It was reestablished as a separate project with its own domain name by January 10. Almost immediately, it began to best its “mother” project: within a year the Wikipedia had published 20,000 articles and existed in 18 different languages; by the Nupedia’s suspension in the fall of 2003, the Wikipedia had published 152,000 articles in English and was found in twenty-six different languages (Multilingual Statistics). By October 30th, 2006, there were over 1.4 million articles in English alone.

The contrasting fates of the Nupedia and the Wikipedia illustrate the central problem that faces any attempt to impose traditional academic structures on projects designed for the participatory web: the strengths and weaknesses of wiki-based and traditional academic models are almost directly out of phase. The Wikipedia has been successful in its quest to develop a free, on-line encyclopaedia of breadth and accuracy comparable to that of its print-based competitors because the barrier to participation is so low. Because anybody can edit the Wikipedia, millions do. And it is their collective contribution of small amounts of effort that enables the growth and success of the overall project.

The Nupedia, on the other hand, failed because its use of traditional academic vetting procedures raised the bar to mass participation by amateurs but did not make the project significantly more attractive to professionals. Academics who need prestige and authorial credit for their professional lives are still going to find it difficult to use participation in the Nupedia (or, now, the Citizendium) on our CVs. Even in fields where collaboration is the norm, scholars need to be able to demonstrate intellectual leadership rather than mere participation. A listing as first author is far more valuable than second or third. And second or third author in a traditional venue is infinitely preferable to professional academics to membership in an relatively undifferentiated list of contributors to an on-line encyclopaedia to which amateurs contribute. The most prestigious journals, presses, and encyclopaedias all enforce far higher standards of selectivity than the mere evidence of an earned Ph.D. suggested by Nupedia and or “eligibility” for “a tenure track job” preferred by the Citizendium. No project that hopes to remain open to free collaboration by even a select group well-informed amateurs or marginally qualified is going to be able to compete directly with already existing, traditional publications for the best original work of professional scholarly researchers, no matter how stringent the review process. But by raising the bar against relatively casual participation by large numbers of amateurs, such projects also risk vitiating the “many hands make light work” principle that underlies the explosive success of the Wikipedia and similar participatory projects.

A New Model of Scholarship: The Wikipedia as Community Service

If I am correct in thinking that attempts to create alternatives to the Wikipedia by combining aspects of traditional academic selectivity and review with a wiki-based open collaboration model are doomed to failure, then the question becomes what “we” (the professional University teachers and researchers who are so suspicious of the original Wikipedia) are to do with what “you” (the amateurs who contribute most of the Wikipedia’s content) produce.

It is clear that we can’t ignore it: no matter what we say in our syllabi, students will continue to use the Wikipedia in their essays and projects—citing it if we allow them to do so, and plagiarising from it if we do not. Just as importantly, the Wikipedia is rapidly becoming the public’s main portal to the subjects we teach and research: popular journalists now regularly cite the Wikipedia in their work and the encyclopaedia commonly shows up on the first page of Google searches. While it may not be in any specific scholar’s individual professional interest to take time away from his or her refereed research in order to contribute to a project that provides so little prestige, it is clearly in our collective interest as a profession to make sure that our disciplines are well represented in the first source to which our students and the broader public turn when they want to find out something about the topics we actually research.

But perhaps this shows us the way forward. Perhaps what we need is to see the Wikipedia and similar participatory sites less as a threat to our way of doing things than a way of making what we do more visible to the general public. The fictional romance between LonelyGirl15 and DanielBeast on YouTube did not threaten the makers of commercial television. But it did give prominence to a medium that makers of commercial television now use regularly to attract audiences to their professional content in the traditional media. In our case, the Wikipedia is less an alternative to traditional scholarship (except perhaps as this is represented in print encyclopaedias) than it is a complement—something that can be used to explain, show off, and broaden the appeal of the work we do in our professional lives.

In fact, the important thing about the Wikpedia is that it has been built almost entirely through the efforts of amateurs—that is to say people who are not paid to conduct research in our disciplines but do so anyway because it is their hobby. While it can certainly be disheartening to see the occasional elementary mistake or outlandish theory in a Wikipedia entry, we should not ignore the fact that the entry itself exists because people were interested enough in what we do to try and imitate it in their spare time. Given the traditional lack of respect shown scholarly research by governments and funding agencies for much of the last century, we should be rejoicing in this demonstration of interest—in much the same way scientists judging a science fair are able to see past the many relatively trivial experiments on display and recognise the event’s importance as a representation of popular interest in what they do.

This recognition of the extent to which the Wikipedia has engaged the imagination of the general public and turned it to the amateur practice of scholarship suggests what I think may prove to be the best way of incorporating it into the lives of professional academics: since the Wikipedia appears unable to serve as a route to professional advancement for intrinsic reasons, perhaps we should begin to see contributions to it by professional scholars as a different type of activity altogether—as a form community service to be performed by academics in much the same way lawyers are often expected to give back to the public through their pro bono work. A glance at almost any discussion page on the Wikipedia will show that the Wikipedians themselves are aware of the dangers posed to the enterprise by the inclusion of fringe theories, poor research, and contributions by people with insufficient disciplinary expertise. As certified experts who work daily with the secondary and primary research required to construct good Wikipedia entries, we are in a position to contribute to the construction of individual articles in a uniquely positive way by taking the time to help clean up and provide balance to entries in our professional areas of interest. In doing so, we can both materially improve the quality of the Wikipedia and demonstrate the importance of professional scholars to a public whose hobby touches very closely on the work we are paid to do—and whose taxes, by and large, support us.

And who knows, maybe “we” could even join “you” in accepting Time Magazine’s nomination for person of the year.

Works Cited

Ahrens, Frank 2006. “A Wikipedia Of Secrets.” Washington Post. Sunday, November 5: F07. Online edition, URL: http://www.washingtonpost.com/wp_dyn/content/article/2006/11/03/AR2006110302015.html

Bails, Jennifer.2006. “Schatten’s hand in bogus paper detailed.” Pittsburg Tribune-Review, January 11. http://www.pittsburghlive.com/x/tribune-review/trib/regional/s_412326.html

Bergstein, Brian. “Microsoft Offers Cash for Wikipedia Edit.” Washington Post, January 23. http://www.washingtonpost.com/wp-dyn/content/article/2007/01/23/AR2007012301025.html

Citizendium 2006. “Citizendium’s Fundamental Policies.” Citizendium (citation from version 1.4, October 11) http://www.citizendium.org/fundamentals.html

Department of English, University of Colorado [2007]. “Department of English guidelines for promotion.” Department Handbook. http://www.colostate.edu/Depts/English/handbook/guidepro.htm

Fung, Brian, 2007. “Wikipedia distresses History Department.” middleburycampus.com. Online. URL: http://media.www.middleburycampus.com/media/storage/paper446/news/2007/01/24/News/Wikipedia.Distresses.History.Department-2670081.shtml

Giles, Jim. 2005. “Internet encyclopaedias go head to head.” news@nature.com. “ http://www.nature.com/news/2005/051212/full/438900a.html”: http://www.nature.com/news/2005/051212/full/438900a.html

Grossman, Lev. 2006. “Time’s Person of the Year: You.” Time. Wednesday, Dec. 13. Online Edition. URL: http://www.time.com/time/magazine/article/0%2C9171%2C1569514%2C00.html .

History of Wikipedia. Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=History_of_Wikipedia&oldid=104389205 (accessed January 31, 2007).

Lonelygirl15. Wikipedia, The Free Encyclopedia, http://en.wikipedia.org/w/index.php?title=Lonelygirl15&oldid=104136723 (accessed January 31, 2007).

Multilingual Statistics. Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=Wikipedia:Multilingual_statistics&oldid=97805501 (accessed February 2, 2007).

Nupedia. Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=Nupedia&oldid=103617050 (accessed January 31, 2007).

Read, Brock. 2006. “Can Wikipedia Ever Make the Grade?” The Chronice of Higher Education October 27. URL: http://chronicle.com/temp/reprint.php?%20id=z6xht2rj60kqmsl8tlq5ltqcshc5y93y

Sanger, Larry J. 2005. “The Early History of Nupedia and Wikipedia: A Memoir.” Part I http://features.slashdot.org/article.pl?sid=05/04/18/164213&tid=95&tid=149&tid=9 Part II: http://features.slashdot.org/article.pl?sid=05/04/19/1746205&tid=95.

Tang, Hangwi and Jennifer Hwee Kwoon Ng. 2006. “Googling for a diagnosis—use of Google as a diagnostic aid: internet based study” BMJ 333)7570): 1143-1145. URL: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1676146: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1676146

Wikipedia: Peer Review. Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=Wikipedia:Peer_review&oldid=104637689 (accessed January 31, 2007).

Wikipedia. Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=Wikipedia&oldid=104645649 (accessed January 31, 2007).

Wired Campus 2006. “Wikipedia Founder Discourages Academic Use of His Creation.” Chronicle of Higher Education. June 12. URL: http://chronicle.com/wiredcampus/article/1328/wikipedia-founder-discourages-academic-use-of-his-creation

----  

Why should I write for your Wiki? Towards an economics of collaborative scholarship.

Posted: Dec 15, 2006 17:12;
Last Modified: Jan 04, 2017 16:01

---

Originally presented at the conference of the Renaissance of America. San Francisco, CA. March, 2006.

I’d like to begin today by telling you the story of how I came to write this paper. Ever since I was in high school, I have used a process called “constructive procrastination” to get things done. This system involves collecting a bunch of projects due at various times and then avoiding work on the one that is due right now by finishing something else instead. Or as my wife, who actually teaches this system says: “if you want me to get your project done today, give me something more important to avoid working on.”

In this particular case, the important thing I wanted to avoid doing was this lecture. And the thing I did instead in order to avoid it was work on an article for the Wikipedia. Or rather—and to be honest, worse—work on revising an article I put up on the Wikipedia almost a year ago when I was was trying to avoid working on an article on Fonts for the Digital Medievalist.

The goal of my procrastination this time was to get my entry recognised as a “Featured article”. A “Featured article” at the Wikipedia is one considered suitable for displaying on the site’s front page. Such articles are supposed to represent the very best of the encyclopaedia, and an unofficial policy, frequently cited by reviewers, restricts them to approximately 0.1% of the total database.

Getting an article recognised as a “Feature” turns out to be a remarkably difficult process. You nominate your work for consideration, at which point it is opened for review by the community at large. And they basically tell you to take it back and make it better. Very few articles seem to sail right through. The ones I saw on their way to featured status had all gone through the process at least once before.

In my case the reviewers didn’t like my referencing style, thought the writing was aimed at too specialised an audience, and generally wanted much more background detail. After two weeks of hard work, and about 100 edits, the article is now beginning to get good rather than lukewarm to negative reviews and now seems on its way to getting recognition as a “feature”. I’m debating resubmitting next time I have something else to avoid doing1.

In addition to being surprisingly conscientious, the comments I received on my piece were also remarkably astute. Unbeknownst to the reviewers, indeed, they accurately reflected the article’s history. I the first added the entry—which is on Cædmon, the Anglo-Saxon poet and subject of my recent book from Boydell and Brewer—last year, when, trying to avoid researching an entry on fonts for the Digital Medievalist, I decided to see how the Wikipedia handled something I knew something about. The original entries on Cædmon and his Hymn were quite inaccurate, and relied on very old sources; the one on Cædmon’s Hymn also made an odd claim about hidden paganism in the poem. In the interests of procrastination, I decided to correct the entry on Cædmon’s Hymn, and replace the account of the poet’s life with an entry I had just written for a specialist print encyclopaedia, The Sources of Anglo-Saxon Literary Culture. With my print editor’s permission, I quickly revised the entry I had submitted to him, cutting out unnecessarily detailed information and adding some background material, and pasted the results into the Wikipedia. There were a couple of minor problems—I forgot to remove some books I was no longer citing from the works cited list, and some of the italics and character encoding were messed up—but on the whole the article received good comments on its discussion page, and was left alone for the most part by other users. This is generally a good sign in the Wikipedia, and in fact a criteria for recognition as a featured article.

My entry for Cædmon’s Hymn didn’t fare as well: the author of the original piece kept reversing my edits until others recommended that the piece be merged with the larger Cædmon article. I never did finish my work for the wiki entry on Fonts that I was supposed to be researching for the Digital Medievalist… though I do have an article due at the beginning of May that I’d like to avoid.

I’ve told you the story of how I came to write this paper—or rather avoid doing so—because I think it illustrates several important things about the possibilities and challenges involved in creating an information commons.

Information commons are a relatively hot topic right now. These are collaborative environments in which content is developed and published interactively—by the users of the community for which it is intended. Such communities can take various forms, but the most common are probably blog farms, RSS servers, Wikis, other types of collaborative tools such as Version Control Systems, annotation engines, and the more familiar chat rooms and email lists.

More importantly, such environments are beginning to become more popular in the scholarly world as well. A number of projects, such as STOA, the Digital Medievalist and Digital Classicist Projects, the Virtual Humanities Lab at Brown, and the Text Encoding Initiative are beginning to use tools like Wikis as a central part of their environment, or experiment with more sophisticated types of collaborative tools such as annotation and editing engines.

What my experience with the Wikipedia shows is that these commons can indeed end up with—if I say so myself—detailed and scholarly work of a relatively high standard. I don’t work for the Wikipedia after all, and I have—for whatever twisted psychological reasons—devoted a reasonable amount of time and expertise contributing a thoroughly researched and referenced entry on my subject.

Moreover, my experience also shows that such communities can be collaborative in the best sense: my article is now much better suited for its intended audience—and better written, I think—as a result of the criticism I received from the Wikipedia reviewers after I nominated it for feature status.

And, a final positive point: it shows that such communities can be self-policing. The person who thought Cædmon was really praising Pagan gods in the original entry (a very non-standard view) was eventually outvoted and reined in by a consensus among other users. And to his credit he accepted this consensus. and moved on.

But my experience also shows some of the difficulties involved in running a community of this sort:

First of all, my best and most careful work appeared only with the prospect of a reward. The reward is not great—neither my dean nor my colleagues are going to care if my article is selected as a “Feature.” But it was only once I decided to go for Feature status that I did the kind of detailed slogging that I normally do in my day-to-day research, and indeed had done in the print entry from which I revised my Wikipedia article.

Secondly, while I did contribute up-to-date scholarship to the Wikipedia, I didn’t do any research for the Wikipedia: I contributed my Cædmon article because I had something suitable lying around which I had already researched and written for a different purpose. Nobody—even the hobbyists who contribute most of the Wikipedia’s material—would put the kind of research I did into a piece written exclusively for it. If they did, it is highly doubtful that they would devote the kind of time to checking citation style and the like that print editors demand from professional scholars.

And finally, although the community is self-policing, it is not always safe to walk the streets at night: the person whose work I corrected, did, after all, come back and undo my revisions. Even though he ultimately gave in to the consensus opinion of the users—and what if the consensus had been wrong?—his inaccuracies nevertheless did replace my corrections for a significant amount of time.

I am not the first person to notice these positive and negative aspects of the commons: having used wikis on a number of projects for a couple of years, I can tell you that the problem of quality control is the second thing most academics comment on when they are introduced to wiki software, after first expressing their admiration for the concept of a user-edited environment. But because these environments are becoming more popular in a scholarly context, it is worthwhile revisiting what are in my view the two most important organisational issues facing scholarly intellectual commons:

  1. How do you get knowledgeable people to contribute their best work?
  2. How do you prevent abuse/vandalism/and/nonsense from the well-meaning but incompetent?

For the rest of this paper, I’m going to address these problems in a fairly speculative way. There are some practical steps we can take right now to find solutions to them but it is worthwhile also thinking about how they might be solved given enough time and technical expertise. Indeed in some ways, my goal is to contribute to a debate in the much the same way one contributes to the Wikipedia: throw something out there and hope that somebody can improve on it.

Although these are crucial problems for intellectual commons, they are by no unique to them. The question of how you get good quality work in and keep the bad out is also central to the operation of peer-reviewed journals or, indeed, any kind of organised communication.

These are crucial problems for an intellectual commons, however, because, in its purest state, a commons has no gatekeeper: the Wikipedia is the encyclopaedia that “_anybody_ can edit” (emphasis added). That is what makes it so exciting but also causes all the problems. Traditionally, scholarly journals and academic presses (organisations that rarely pay their authors) have addressed this problem with a combination of carrots and sticks: they encourage people to contribute by providing enough prestige to make it worth their while to submit well researched articles, and they keep the bad stuff out by getting disciplinary referees to review the submitted articles before they are printed.

A true intellectual commons lacks both a system providing rewards and preventing folly. Perhaps for this reason, most academic commons rely on some kind of gatekeeper: you need to be approved by a moderator if you want to join a mailing list; you need to submit a CV if you want to be able to annotate an online edition; you need to have your login approved by a Sysop if you want to contribute to a scholarly wiki. Even beyond this, such projects also usually engage in editorial control: spammers are cut off, trolls and flamers are banned, and wiki or annotation contributions are reviewed for quality by some central person or group.

These approaches are effective on the whole at preventing or mitigating abuse by unqualified or out-of-control people. They do, however, suffer from two main problems:

  1. They scale very badly: while a gate keeper or moderator can vet or edit contributions from a small number of people, this gets progressively more difficult as the community expands.
  2. They represent a compromise on the thing that makes commons different and exciting in the first place: the possibility for unnegotiated collaboration and exchange.

Scaling is probably not an issue for most academic projects. Digital Medievalist is a relatively big project now, for example, and it is only approaching 250 members. Numbers like this are relatively easy to control. The costs one would incur in trying to develop an automatic vetting system for a market this size would vastly outweigh any future benefit.

Other disciplines, however, have been faced by this scaling problem—and managed to find partial solutions that in my opinion do a better job of maintaining the unnegotiated quality that make successful commons what they are.

One solution commonly proposed solution is to rely on distributed moderation—or, in simple terms—allow the users to police themselves. This has the advantage of being completely scalable—the number of moderators increases with the number of users. As we saw in my experience with the Wikipedia, moreover, this system actually actually can work: many (perhaps most) errors on the Wikipedia are corrected after a while and unqualified or insincere contributors often do get reined in.

But of course my experience with the Wikipedia also shows the problem with this approach. If everybody can be a moderator, then the unqualified can be as well. They can, as a result, replace good work with bad as easily as others can replace bad work with good.

A solution to this problem is to allow moderation only by respected members of the community. This is the system at Slashdot.org, a newservice for technological news. There contributors acquire a reputation based on other’s opinions of their contributions; those with high reputation scores are then added to a pool from which moderators are drawn each week (the system is actually much more complex, but the details are not important here).

Systems such as this tend to suffer from complexity: Slashdot also has meta-moderation and nobody seems very happy with anybody else even then. Moreover, members have a tendency both to game the system in order to increase their own reputations and lower those of their “enemies”.

At Digital Medievalist, we have been thinking of a slightly different model of distributed moderation, which we describe as an apprenticeship model: in this solution, newcomers are assigned relatively limited editorial, moderation, and compositional powers. These powers then increase as one’s contributions are allowed to stand by other members of the community. Initially, one might be allowed only to correct typos; as people accept your corrections, you are allowed greater editorial powers—perhaps you can rewrite entire sections or contribute new articles. If, however, your contributions begin to be rolled back, your powers shrink accordingly: the idea is ultimately a version of the Peter Principle: you rise to the point at which you are perceived to become incompetent. The main difference is that we then try to push you back down a step to the last place in the hierarchy in which you knew what you were doing.

This method would require considerable software design, and so, currently, is outside our ability. It would have the advantage over the Slashdot method, however, both of scoring ‘reputation’ on the basis of audience’s real behaviour (reducing your ‘enemy’s’ score requires you to take the time to reverse his or her edits?) and of keeping track of reputation not by points (which encourage people to be competitive) but by permissions. A points system encourages people ask themselves how much they are worth; a permissions system encourages them to take on extra responsibility.

Moderation systems are essentially negative: they exist to prevent people from messing things up. As I noted earlier, however, commons also have the positive problem of trying to encourage good work: the most strictly refereed journal in the world, as the PMLA discovered a few years back, is no good if nobody submits articles to be vetted.

This is an area in which scholarly projects seem to have devoted less attention. While most projects with commons-type environments have explicit moderation policies, few if any I have seen have explicit reward policies. They tend to have gatekeepers but no paymasters. Perhaps as a result most also seem to be the work of a very small number of people—even in the case of organisations with large numbers of members.

Once again, solutions for this problem can be found in other disciplines. The Open Source software movement, for example, relies on high quality contributions from volunteers. Open Source groups often reward significant contributors by treating work on the project as a form of “sweat equity” that allows them special privileges: eligibility for membership on the executive or board, for example, or voting rights or even basic membership.

A second solution commonly used is to give significant contributors some kind of token that sets them apart from others. This can be as valuable as the right to administer or moderate others (Slashdot), or as minor as extra stars beside your user name in the forum (Ubuntu).

Both of these solutions can be adapted to the scholarly commons. At Digital Medievalist, we are currently putting together new bylaws that will treat contributions as a condition of membership: people who contribute to the wiki, mailing list, or journal, will be eligible to assume responsibility as board members or officers of the project (I suspect giving away extra stars for good contributions might not be as effective in the academic world as it seems to be in the Open Source one—though given how important psychologically such distinctions are, perhaps they would). A second possibility for reward—albeit one fraught with difficulties—might be to award named authorship of material on which an contributor has worked: either by naming people at the top of the article or adding them as contributors to a colophon on the project as a whole.

The two easiest solutions to this problem of reward, however, are probably those used by the Wikipedia to get me to revise my article on Cædmon rather than work on this talk: offer special status for particularly well done work, and design the project as a whole so that it is a natural outlet for work that otherwise might not be used. At the Digital Medievalist, we already run a peer-reviewed journal alongside our wiki-based commons. A project with a different focus might certify certain articles in some way: as “refereed” vs. “open forum”, perhaps, and identify the authors and major contributors. Our project, moreover, is set up to provide a forum in which users can publish material they might otherwise find hard to use in furthering their careers: solutions to technical problems in digital humanities such as the development of stylesheets and databases, that are not commonly published by the major disciplinary journals.

The intellectual commons represents a new, purely digital approach to publication and the dissemination of scholarship. It is a model that cannot be replicated in print, and it is a model that many scholars feel intuitively at least will become a major force in the future of scholarly communication. In order for it to realise its potential, however, we must first find an economic model that encourages us to contribute our best work and provides for some kind of quality control—without sacrificing the very spontaneity that defines this new means of communication.

So why should I write for your Wiki? Until we answer this question, the Wiki will not live up to its full scholarly potential.

1 Update: The entry ultimately gained feature status.

----  

Back to content

Search my site

Sections

Current teaching

Recent changes to this site

Tags

anglo-saxon studies, caedmon, citation, citation practice, citations, composition, computers, digital humanities, digital pedagogy, exercises, grammar, history, moodle, old english, pedagogy, research, student employees, students, study tips, teaching, tips, tutorials, unessay, universities, university of lethbridge

See all...

Follow me on Twitter

At the dpod blog