I’ve been somewhat irritated recently, as I’m sure most academics have been, by the increasing interest that our universities have in citations. Citations count in league tables. We – academics – are increasingly told that we must focus more on how to ramp up our own citations. It’s recently been explained to me for example that citations can be increased by writing review articles and by writing with colleagues from particular US and Australian institutions.

This kind of advice is invariably based on interpretations of citation metrics. I strongly suspect that these explanations of how-to-write-a-highly-cited- paper are a case of people (who ought to know better) mistaking correlations for causes. Apart from the times when academics cite papers that are bad, or because they are written by someone already famous, my sense is that it’s more generally the case that the papers that are cited a lot also happen to be significant in some way. That’s not to say that all significant papers get cited a lot, but rather that if a paper gets a lot of citations it’s because colleagues are referring to what it says for a reason. Well, that’s my hypothesis anyway.

So how would I test this out? Well, I think it might actually be helpful to have some further research and analysis. Maybe we could actually look at some highly cited papers to try to understand better why they have become important in their field. It might also be interesting to talk to their authors and some of their readers, those who do the citing.

Let me give you an example to show why I think that such an endeavour might be of use.

A little while ago, and in my own field of education, an editor of the journal Discourse asked an author – Stephen Ball – to look back at, and talk about, what is now a very highly cited paper. It was something that he’d written twenty one years previous. Ball recorded his reflections on What is policy: texts, trajectories and toolboxes (passworded) in a Youtube clip.

Ball pointed to the paper as a transition point in his own thinking. Ball suggests that the paper was intended to provoke peers and to create a new critical basis for policy analysis – to provide “more theoretical heft”. The paper not only marked a profound shift in his own research and provided the basis for his next two decades of subsequent inquiry, but also laid out a new agenda for the wider field. Ball had an ambition for the paper, beyond his own thinking and work, which went to the state of scholarship at the time. He wasn’t thinking about getting citations, but about purpose and influence. He was concerned about getting read but not about measurement.

This is a particular case, of course, but what’s most interesting to me is the way in which Ball is able to talk about his own research trajectory at the same time as making clear his ambition to make a very significant contribution to the field in which he was researching. He thought the field was limited, and he wanted to change it.

Even this one example sheds a little light on one way that important contributions to the discipline and to the field can be made through an individual intervention. This single example illustrates a way in which making a significant contribution might be imagined and accomplished. Personal research and change in the field can happen together. Getting a highly cited paper in the field is a by-product, not the point, of the publication exercise.

My hunch is that something like Ball’s example may well be the case with a lot of highly cited papers. I can’t really believe that the papers that everyone refers to have been developed by people saying to themselves – Now I need to get a highly cited paper, I’m going to co-write a review paper with a colleague in a university above mine in the league tables. That’ll make sure I get a big hit in the indices. NOT. I suspect it’s much more likely to be the case that the highly cited paper comes from a happy meeting of luck, the zeitgeist and a researcher thinking seriously about their own scholarship in the context of the current state of the field.

In other words, I strongly suspect that the bulk of high citations are connected to scholarship, and are not a highly calculated and self-serving act.

I’m pretty convinced that we need something in addition to metric analysis to actually understand citations. Don’t we all teach research methods courses which say that numbers can do some things and not others? …. But of course we need many more than this one example I’ve given. N = one doesn’t fly either, even if it is interesting. It might be enough however to suggest the merit in looking beyond numbers when we think about influencing the field and getting readers.

But, in the meantime and in the absence of research more thorough than that available from indices, I’m going to take those metric-based causal analyses about how I should write, about what and with whom, with a very large grain of salt.

Author Bio: Pat Thomson is Professor of Education in the School of Education, The University of Nottingham. Her research is centred primarily on how schools might change to be more engaging and meaningful for more children and young people.