Rank delusions



Every year, U.S. News & World Report, Times Higher Education, and others update university rankings. Reactions are paradoxical. On the one hand, university administrators and faculty members scan the lists for evidence of small movements up or down. On the other hand, everyone knows that the top 10, or 20, or 50 names will be much the same as they have always been.

The Duke sociologist Kieran Healy points to a four-tier classification of leading universities made in 1911, and compares it to the most recent U.S. News ranking. Of the top 20 universities in the ranking today, 16 were in the top class in 1911, one (Notre Dame) was in the second class, and three (Duke, Rice, and Caltech) had not yet been established under their current names.

The United States is not unusual in this respect. In most countries with an established higher-ed tradition, the list of high-status universities has changed little over decades or even centuries. There have been some modest shifts in the relative status of different kinds of universities (for example, private versus public in the United States), and there are some impressive new institutions in Asia and other areas of rapid economic growth, but the impacts are marginal.

Broadly speaking, the 1911 list would not raise any eyebrows if it were used as the basis for the next U.S. News ranking. For those seeking to answer the question \”What makes a great university?\” the answer appears to be \”having been great 100 years ago.\”

Now compare the Dow Jones Industrial Average for 1911, which included such companies as the American Smelting and Refining Company (now Asarco), U.S. Rubber (now Uniroyal), and U.S. Steel. Some of those companies have vanished altogether, and others have survived as subsidiaries of a larger enterprise, but only General Electric is still included in the Dow Jones index. Most of today’s Dow Jones companies did not even exist in 1911.

What accounts for the remarkable stability of university rankings?

What accounts for the remarkable stability of university rankings in comparison to the instability of big business, and for that matter, other nonprofits? More important, what implications does this have for university management and higher-education policy?

Several features of universities are important in explaining these outcomes. First, unlike other enterprises, universities almost never die and rarely merge. The 14 universities that formed the Association of American Universities, in 1900, are all still in existence, as are all those admitted since then.

Second, and directly related, universities are what are called, in the literature on industrial organization, \”single-plant firms.\” The vast majority have one (or at most two) main campuses, with a few peripheral offshoots. Apparent exceptions like the University of California system are in reality a set of distinct universities, linked only by notionally shared governance.

Those structural facts put an upper bound on the feasible size of a university. A single campus can’t accommodate more than about 40,000 undergraduate students without running into diseconomies of scale, such as constraints on the size of lecture halls. The biggest state universities reached that size in the 1970s, and their enrollments have remained broadly stable ever since. Elite private universities operate on a much smaller scale, typically 3,000 to 5,000 students, and most have maintained that size since the 1950s.

Taken together, those facts rule out many of the mechanisms by which markets reward success and punish failure. A successful university doesn’t typically create new campuses or even greatly expand its enrollments. (Some American universities are attempting to test that proposition by establishing offshore campuses such as those of Yale in Singapore and NYU in the UAE. If only they’d learned from the experience of their Australian counterparts who followed the same path in the 1990s, with results ranging from disappointing to disastrous.) Conversely, poor performance may create stresses of various kinds, but almost never leads universities to close down, or even to radically contract.

As a result, growth in the university system has occurred primarily through the creation of new universities, or through the upgrading of vocational-training institutions such as teachers’ colleges. At least initially, the new entries are almost always at the lower levels of the status hierarchy. The creation of a new research university, such as the University of California at Merced, is a rare event.

Those facts are enough to explain part of the difference between the relatively stable ranking of universities and the ever-changing rankings of top companies. With no departures, and limited possibilities for growth, the only way that universities can change their ranking is through a change in the (perceived) quality of their research and teaching. This is necessarily a slow process.

Moreover, universities are nonprofit enterprises that nonetheless generate substantial operating surpluses. In the absence of shareholders, the surplus generated by a university is available to improve the university’s standing, for example by hiring star professors, establishing new research centers, or adding facilities to attract students.

But while those structural features explain why the relative status of universities doesn’t change much from year to year or decade to decade, they don’t explain the near-constancy of the rankings over scales of a century or more.

In statistical terms, we can think of university status as a process characterized by mean reversion. That is, if a high-status university performs poorly for some time, perhaps because of poor leadership or bad hiring decisions, it is likely to recover the lost ground over time. Conversely, a lower-status university that does well for a few years will find it difficult to maintain its enhanced status.

The crucial factor in explaining mean reversion is the existence of exceptionally durable assets of various kinds, the most important of which are human and reputational. A long-established high-status university has a large body of alumni, Ph.D. graduates, former faculty members, and research collaborators. Apart from obvious benefits such as alumni donations, that group can be looked to as a source of legacy students, opportunities for graduate placements, and senior hires keen to return to their former affiliation.

One way to test that idea is to look for other examples of status competition where rankings remain stable over long periods. An interesting case is that of European soccer leagues. Unlike American sport leagues, these mostly lack a draft or salary cap. Mobility is supposed to be achieved through a system of promotion and relegation, in which the winners in lower-division competitions move up, while the bottom-placed teams in the higher division move down. In practice, however, promoted teams usually struggle, while those relegated one year often return to the higher division the next.

Moreover, while most teams are privately owned, few of the owners seek to extract profits. Rather, returns are plowed back into the team and used to attract better players. That in turn produces winning records, which attract more fans and more revenue. Once attracted, fans, like alumni in the university context, are commonly lifelong assets.

Not surprisingly, the results are the same as in competition between universities. Most of the European leagues, notably including those of Italy, the Netherlands, and Spain, are dominated by the same two or three clubs, decade after decade. The Scottish competition provides a typical, if somewhat extreme, example. In 118 years of competition, two clubs (Rangers and Celtic, known collectively as the Old Firm) have won 99 times between them.

The Scottish League also provides a good example of mean reversion. A financial scandal in 2012 led to the Rangers’ being declared insolvent and ejected from the competition. A reformed club was admitted to the Third Division (roughly the equivalent of the bottom tier of minor-league baseball), but with its top management, most of its money, and many star players gone. Crucially, however, the new Rangers retained the membership and fan base of the old one. It is rapidly ascending the status ladder and is set to resume its rivalry with Celtic in a season or two.

What should we learn from all this?

Most obviously, there is not much point in worrying about university rankings, whoever may issue them. Differences from year to year in a given ranking, or between different rankings, will inevitably be dominated by random noise. But even if changes in rankings reflect actual differences in performance, mean reversion ensures that this will mostly wash out over time. For the kinds of decisions for which rankings should matter, such as which university to attend, year-to-year variations are of no significance.

Second, it seems unlikely that university presidents, or other top managers, can make much difference in the way their institutions perform. As much as a decade at the helm is still simply too short to produce any sustained shift in rankings. Conversely, the departure of presidents and other top administrators, even under a cloud of disgrace, seems to have little or no impact on the status of the institutions concerned.

But the big question is whether the stable hierarchy is beneficial or harmful to the teaching and research mission of the university system as a whole. If harmful, what can be done about it?

As regards research, the advantages of stratification are obvious. The institutions at the top of the status hierarchy have continued to produce the bulk of research in leading journals, to earn Nobel Prizes and similar awards, and so on. Nevertheless, there are plenty of cases where this self-perpetuating elite might benefit from the challenge of outside perspectives.

For undergraduate education, American experience has shown that a highly stratified system works poorly. Competition for status encourages high-ranked institutions to restrict enrollments and to provide a high-quality experience to a small number of students, which inevitably implies high tuition fees. Since new entrants to the system inevitably enter with lower status, a steep and stable hierarchy implies that an increasing proportion of students attend poorly funded institutions that struggle to provide a quality education.

What, if anything, can be done to flatten the hierarchy and increase mobility within it? Sporting leagues have adopted solutions such as salary caps and draft systems for the recruitment of new players. Analogs could be imagined in the university context, but seem unlikely to command much support.

A more plausible response is a shift in public-financing priorities. If, as we’ve seen, success or failure in the status race is largely preordained, financing systems designed to reward and \”incentivize\” success are misconceived. Support should be allocated on the basis of need rather than used to amplify historical advantage. In this respect, President Obama’s initiative to expand access to community college is a step in the right direction.

Author Bio: John Quiggin is a fellow in economics at the University of Queensland; a columnist for The Australian Financial Review; a blogger for Crooked Timber; and the author of Zombie Economics: How Dead Ideas Still Walk Among Us (Prince­ton University Press, 2010).