Can Metrics Be Used Responsibly? Why Structural Conditions Push Against This

Not waving, exactly, but...

Not waving, exactly, but…

Today, the long-awaited ‘Metric Tide’ report from the Independent Review of the Role of Metrics in Research Assessment and Management was published, along with appendices detailing its literature review and correlation studies. The main take-away is: IF you’re going to use metrics, you should use them responsibly (NB NOT: You should use metrics and use them responsibly). The findings and ethos are covered in the Times Higher and summarised in Nature by the Review Chair James Wilsdon, and further comments from Stephen Curry (Review team) and Steven Hill (HEFCE) are published. I highly recommend this response to the findings by David Colquhoun. You can also follow #HEFCEMetrics on Twitter for more snippets of the day. Comments by Cambridge lab head Professor Ottoline Leyser were a particular highlight.

I was asked to give a response to the report at today’s launch event, following up on the significance of mine and Pablo’s widely endorsed submission to the review. I am told that the event was recorded by video and audio so I will add links to that when they show up. But before then, a short summary record of the main points I made: Continue reading

One Size Fits All?: Social Science and Open Access

The third post in our small series on open access, publication shifts on the horizon and how it all matters to IR and social science, this time by David Mainwaring  (Pablo’s post was firstthen Colin Wight’s, and following David came Nivi Manchanda, Nathan Coombs and our own Meera). David is a Senior Editor at SAGE with responsibility for journals in politics and international studies. So he oversees journals like Millennium and European Journal of International Relations amongst others (which is how we know him), and can thus offer a close reading of movements within the academic publishing industry. Images by Pablo.


Open Access is the talk of the academic town. The removal of barriers to the online access and re-use of scholarly research is being driven by a cluster of technological, financial, moral and commercial imperatives, and the message from governments and funding agencies is clear: the future is open. What is much less clear is exactly what sort of open future social scientists would benefit from, let alone what steps need to be taken in order to transition away from the existing arrangements of scholarly communication and validation. Here the conversation is in its relative infancy, characterised at this point by a great deal of curiosity, anticipation, confusion, and the shock of the new. What it needs to move towards is a recognition and coordinated response to the fact that although social science may share the same open access goal as the STEM disciplines, the motivations for travelling down that path are not identical, and the context – especially in terms of research funding – is significantly different. The roundtable discussion at the Millennium conference at the LSE on 20th October was an attempt to explore these issues specifically from an IR perspective; further such events (such as those being run by the AcSS and LSE this autumn) are to be warmly welcomed as a means of building broader understanding of the issues among social scientists and facilitating strategic thinking.

Social Science and the Open Access Debate

The long-running debate about how scholarly research communication should be funded and transmitted has been, and remains, a discussion conducted primarily by those working in STEM. Most of the blogosphere’s best-known voices on open access, the likes of Mike Taylor, Michael Eisen, Peter Murray-Rust, Björn Brembs, Cameron Neylon, Kent Anderson, Stephen Curry and Tim Gowers, have backgrounds in STEM research or publishing. (Notable exceptions are the philosopher Peter Suber, who heads the Harvard Open Access Project, and self-archiving advocate Stevan Harnad, a cognitive scientist). To dip into the often heated debates on open access can leave you with the strong impression that, despite the occasional nod to social science and the humanities, the frame of reference is proper, rigorous, natural scientific research, the kind carried out in a laboratory that leads to medical advances and the development of new technologies.

By and large social scientists – and arts and humanities scholars, to whom many of the points raised in this piece apply equally – have had a back seat in this conversation, and the development of open access awareness and capabilities has been slow. Many leading social science journals continue to be distributed in print form due to subscriber demand well over a decade after the launch of their online editions. The American Political Science Association’s 2009 book Publishing Political Science devotes just two out of over 250 pages to open access, and fewer than 10% of the nearly 13,000 signatories of the ‘Cost of Knowledge‘ boycott of Elsevier were social scientists, despite the company’s position as one of the world’s leading publishers of social science journals. That’s not to say that there is nothing going on: the Social Science Research Network has acted as a site for open paper sharing since 1994; there are active ‘open’ movements in disciplines including IR and economics; and the Directory of Open Access Journals lists more than 1,600 social science titles. To date, however, very few of the latter have been able to break into the higher echelons of profile or reputation within their fields.

Social Science and Open Access Mandates

Over the last eighteen months a series of events, including George Monbiot’s polemic in The Guardian and the defeat in the US of the Research Works Act, stirred for the first time a significant consciousness among social scientists about open access. This awareness increased dramatically – in the UK at least – with the publication in June 2012 of the final report of a committee set up by the Government to examine access to published research findings. Continue reading

Death To Open Access! Long Live Open Access!

A few weeks ago at the Millennium conference, some of us got together to talk about open access and the political economy of knowledge (re)production in our little corner of academia (“us” being Colin Wight, David Mainwaring, Nivi Manchanda, Nathan Coombs, Meera and me). Over the remainder of this week, we’ll be posting those reflections here for your delectation because, some discussion notwithstanding, labourers in today’s university-factories need to get talking about these things, and fast.


Open Access appears to be here. The Finch Report has recommended it, the Government has endorsed it, and there even seem to be some monies newly available for it. The battle is won, and the age of unfettered academic-public intercourse is upon us. Well, not quite. Finch’s preference for Gold Open Access, in which journals continue to receive revenues and make profits and in which academics (or their institutions) pay a fee of several thousand pounds per article for the pleasure (the so called Article Processing Charge (APC) system, which will receive greater attention in later posts), is deeply problematic (well-reasoned explanations for why available from Stevan Harnad and Peter Coles (Telescoper), with more qualified views, even cautious support, from Stephen Curry and Repository Man). Also, the monies aren’t new, and have instead been extracted from existing research budgets (and what a complete and utter surprise that is).

This is all cause for serious concern, and relates closely to the kinds of arguments that are developed and deployed in favour of Open Access (or, to be more provincial about it, Open IR). There are three kinds of arguments for opening up the journal system, arguments from access, ethics and cost, and we are in danger of letting the first overwhelm the different, and better, cases made in the name of the second and third.

The first set of arguments has to do with the problem of access: that the journal system is broken because it creates barriers to the circulation of academic knowledge. The journal as usually conceived was an ingenious and appropriate method for collating and distributing knowledge in the 18th century, but is now redundant. And yet journals – with their pay walls – remain the only (or at least the principal) vector for success or employment in academic life. This is doubly problematic since the various metrics that journal comparisons and prestige enable themselves then become vectors of discipline and control, even in the face of the many, many reasoned objections to such measures.

Continue reading