University Ranking : Diversity, Excellence and the European Initiative - par Geoffrey Boulton, LERU, juin 2010

LERU signifie League of European Research University et rassemble Universiteit van Amsterdam - Universitat de Barcelona - University of Cambridge - University of Edinburgh - Albert-
Ludwigs-Universität Freiburg - Université de Genève - Ruprecht-Karls-Universität Heidelberg - Helsingin yliopisto
(University of Helsinki) - Universiteit Leiden - Katholieke Universiteit Leuven - Imperial College London - University
College London - Lunds universitet - Università degli Studi di Milano - Ludwig-Maximilians-Universität München -
University of Oxford - Université Pierre et Marie Curie, Paris - Université Paris-Sud 11 - Karolinska Institutet, Stockholm -
Université de Strasbourg - Universiteit Utrecht - Universität Zürich
LEAGUE OFEUROPEAN RESEARCHUNIVERSITIES

Geoffrey Boulton est professeur à Edinburgh University, G.Boulton@ed.ac.uk

Veuillez télécharger le document joint pour lire cet article, qui discute deux initiatives de la Commission européenne
visant à mettre en place un "autre" classement des universités et a suscité un article et deux courriers dans le Times Higher Education .

1) ’Serious defects’ apparent in ’crude’ European rankings project
24 June 2010, By Hannah Fearn

2) Spice of academic life - 1 July 2010

« We were a little surprised to see how our U-Multirank project was described in THE ("’Serious defects’ apparent in ’crude’ European rankings project", News, 24 June)... »

3) Category and other errors - 8 July 2010

« The headline "’Serious defects’ apparent in ’crude’ European rankings project" and its associated news story suggest that your staff had not read the paper at the heart of the story (News, 24 June). (...) »

Résumé en anglais :

• International rankings of universities influence the
perceptions and priorities of governments, of busi-
nesses and students. Rectors and university councils
see the achievement of high ranking as a strategic
imperative.

• However, their value and benefit is questionable.
The fundamental problems are two-fold :
 Most seek to capture characteristics that cannot be
measured directly, and require indirect proxies.
How good are the proxies ?
 Different universities fulfill different roles, which
a single monotonic scale cannot capture. How can
different roles be compared in meaningful ways ?

• None of the current ranking systems have the validi-
ty, rigour or meaning to be of real value, except those
based on citations to evaluate research, and even
here, they fall short in assessing research in the
humanities and the social sciences.

• Institutions tend to target a high score irrespective of
whether the metrics are good proxies for the under-
lying value of the institution. Rankings will at best be
irrelevant to those values or, at worst, undermine
them. They encourage convergence towards a
research-dominated model, reducing system diversi-
ty and undermining the potential to contribute to
society in other ways.

• But rankings have such a hold on the public imagi-
nation that they are likely to be permanent features
of the landscape. Can they be improved ? Two
approaches have been funded by the European
Commission.

• U-Mapis an attempt at classification describing the
diversity of universities by mapping activities, not
quality : its purpose being transparency for stake-
holders.

• U-Multirankis an attempt at ranking evaluating
quality in dimensions analogous to those of U-Map :
its purpose assessing how well universities perform
their different roles, rather than holding all to
research-dominated criteria.

• Both have serious defects. They suffer from impre-
cise proxies and the profound difficulty of finding
comparable data between countries. The tempta-
tions will be to :
 require ever more burdensome detail in the hope
of penetrating to the heart of the matter,
 formalise the distinctions that mapping reveals,
 promote further the idea of the university as mere-
ly a source of modular products currently in
vogue.

• LERU applauds the attempt to create U-Map as a
description of diversity, but is less enthusiastic about
U-Multirank, because of problems of data compara-
bility, the potential for game playing when reputa-
tions for excellence are at stake, and for the encour-
agement it gives to target proxies rather than under-
lying reality.

• Although the U-Multirank approach at least provides
a means of exemplifying diverse forms of excellence
as an antidote to single monotonic lists, it is
inevitable that its individual dimensions will be com-
bined by others to create a single monotonic table of
excellence, regardless of the strictures of its authors.

• However, given the likely persistence of “league
tables”, LERU supports the Commission’s initiative
to develop U-Multirank as a pilot project and as a
means for exploring its potential to mitigate the
problems of other systems. In this spirit, a number
of LERU universities have agreed to collaborate in
the project with the intention of improving it as far
as is possible.