Open Source Peer-Review
February 16th, 2009Scientific journals have genuinely got the best business model in the history of anything ever. Here is how it works, in a nutshell:
- Other people, scientists, write their content for no fee.
- The journal then gets other scientists to review it. These scientists generally don’t get paid either.
- The authors edit the paper and send it back. Eventually, all the scientists reach a version of the paper they can all agree on (or the paper gets withdrawn). Then the authors pay the journal to publish the paper.
- The journal then charges anyone who ever wants to read the paper extortionate fees. $50 for a PDF file is not uncommon.
- The journal retains the copyright on the words they didn’t write describing experiments they didn’t do, and claim fees for reading it at least until the copyright expires and usually long after that.
- None of the scientists or their employers ever get paid.
This, to me, seems like an insane system. It survives because universities don’t care if they pay extortionate fees for such things and because it’s established. And probably when it was established it made sense — after all, who else but journals could publish things? But now it’s just academia needlessly funnelling money into a mostly pointless publishing racket. I really don’t see what it achieves.
I’ve said before that a better system would just be for universities to publish papers and let anyone who wants to comment on them comment on them. I hadn’t worked out the details, obviously, so it was one lunchtime rant in the pub opposite the lab and I didn’t think much more about it until the next time I hit a paywall demanding I give some publishers $75 to read a paper written by my own supervisor. But someone else has worked them out. It’s not perhaps an ideal system, but it looks pretty good to me and it’s compatible with the existing system.
Public Key Cryptography for those who haven’t heard of it.
You have a public key and a private key. You can encrypt something with the private key and it can be decrypted with the public one, so you can use it to prove that you wrote it. I think you can also encrypt something with the public key that only the holder of the private key can read. It’s basically just magic.
Dubbed GPeerReview (I don’t know what the ‘G’ stands for but the author’s name is Gashler so that’s likely), the idea is that you post your paper on your academic website, email people you think would be interested, and those and any other readers can review it. They sign their review, along with a hash of the paper, with public key cryptography so you know who wrote it and what about. That way, you get an idea of how much support a paper has and, crucially, what kinds of people support it. The author of the paper puts up the most credible supportive reviews they can find. In theory, if it becomes accepted then there’ll no longer be any need for conventional publication. It’s a very clever system. (See also, the more established ResearchBlogging.org – which Gashler says could be complementary to GPeerReview but covers rather different ground. I think I agree with him on that — it’d be great to see things like that running it tandem.)
I’d love to see something like this made to work across academia. I suspect, though, that what kills it will be that real people don’t understand nerd stuff like public key cryptography. Everyone else in my research unit gets all annoyed if I try to use LaTeX or Bibtex at them. (Well, the dentists do — the other physicists love it. I witnessed a long argument about a week ago over the relative merits of Microsoft Project versus the open source alternative, which boiled down in the end to ‘well the free software probably is better but if we collaborate with anyone else they’ll demand we use Project’ which to me seems like a really crappy way of doing things — I’d rather piss people off by doing the right thing than pander to idiots and help keep Microsoft’s monopoly on proprietary, buggy software healthy.) They act as if Word and EndNote are somehow better. In my experience, Word doesn’t work properly and EndNote formats citations basically at random. LaTeX is a pig to get set up but at least once you’ve done it it stays set up. To be honest, I think that’s another thing that needs sorting: we need a specialist scientific markup language. Maybe a form of HTML (or other XML), with a standard equation format and a few extra specialised tags, perhaps including COinS for citations, which the reader software could be configured to render as a conventional reference, or as a hyperlink, or as whatever they like. A CSS-like ‘default’ style for a particular paper would be fair enough, but the current system that forcibly changes the format depending on which journal happens to have published the paper is rather silly. I don’t want a stack of PDF files all formatted differently. I want a folder full of pictures and ASCII-encoded markup that I can process and output how I like. Get into the twenty-first century. That’s how we do things here, because it’s a better way of doing it.
And there’s no reason that all of the above couldn’t be implemented really very easily, and I’d love to see peer review evolve into something more open and transparent than the existing system, which still relies on the trustworthiness of journal editors and the word of a few unidentified reviewers per paper. But we need nice, simple user interfaces on every part of it or else Joe Scientist isn’t going to actually bother to do it. We need a nice WYSIWYG program to edit the papers, then a nice Wordpress-style package to maintain your site, and a nice package to let you write reviews without much effort. Make it simple, and people might adopt it. Which is frustrating, because by rights you’d think a good scientist would be exactly the kind of person who would leap at the chance to adopt an open, collaborative, technological and free solution to a problem. Those are the qualities that science runs on. And I can’t see what we’d lose by switching to such a system, other than a load of jobs at journal publishers — and I’m sure the big journals would find a way to adapt. Perhaps they’d act as aggregators or run interesting comment pieces more often or something. (I should link to this very interesting discussion, where Gashler explains what journals do that is useful and that GPeerReview doesn’t do. I’m not convinced it’s all really a job for journals per se, but someone will have to keep doing all that. Personally, I think universities should do most of it.)
Bah. I just get frustrated when people cling to what they know instead of adopting obviously better alternatives, like Linux or metric or atheism or not torturing people. I guess that’s just a failing I have. But I’d love to know how any of the above could be shoehorned into the modern scientific community.
Tags for this article: GPeerReview
[?][More Help]
February 16th, 2009 at 02:18
There’s MathML for representing equations through markup, although it can get very hairy very fast.
February 16th, 2009 at 11:07
Yes, I’m aware of that. LaTeX has a system, too, which I’m given to understand is better although I really don’t know why it is. It’s less ‘HTML-ey’ but I don’t think that’s important.
February 18th, 2009 at 16:58
Dear Andrew,
Perhaps you should become involved in this:
http://www.epsrc.ac.uk/CallsForProposals/dereoicall