Universities and research have been the subject of reforms and evaluations for twenty years. Clémentine Gozlan studies the making of these devices in the new government of science in which many researchers themselves are involved.
Numerous reforms aim to transform the higher education and research system since the 2000s. These have given rise to as much work concerning the transformations of the modes of governance of professions, the recompositions within professional groups, the effects of evaluation on the production of knowledge and the ways in which academics appropriate rankings and indicators. Faced with this, the originality and interest of Clémentine Gozlan's work is to discuss the role of researchers in the design of managerial and political instruments and to study the making of the devices rather than focusing on the effects of reforms.
Evaluation is a familiar activity for researchers, part of scientific practices: it involves both the evaluation of publications by peers and the evaluation of laboratories and research centers by specific bodies. However, the creation of the Research and Higher Education Evaluation Agency (AERES) in 2007 sparked a lot of debate and reluctance in the scientific community, which the author explains by the major changes introduced in the government of science.
Based on this observation, the author proposes to study this institution as an observatory of controversies and power struggles in the scientific community, a place where professional standards and judgment practices are constructed, during its five years of existence (2007-2012). ). To do this, the survey focuses on one of the three sections of theAERES (that dealing with the evaluation of laboratories or research units) and on the human and social sciences (HSS), areas less covered by work on the history and sociology of science (Lamont, 2009). In these disciplines, the introduction of new evaluation methods has been the most controversial (Aust, Gozlan, 2018), which the author explains by the fact that certain evaluation instruments go against conceptions of quality in research (for example, project funding or rankings). The investigation consists of in-depth interviews, direct commission observations, supplemented by analyzes (qualitative and quantitative) of numerous written sources.
The issues raised by this work are multiple, we will detail three of them here: the design of evaluation tools, their contestation and the appropriation that evaluators make of them.
Construction of the agency and design of evaluation tools
The survey allows us to go beyond the idea according to whichAERES would be the result of a reform imposed by the top to academics, although the creation of the agency was a political initiative. Even if new actors outside the scientific profession (managers, senior civil servants, consultants, etc.) are involved, the author shows that teacher-researchers were central in this process, which allows her to affirm that the agency is neither a transmission belt of the state which through it governs the scientific community. (p. 42), nor entirely subject to political control.
The contribution of this demonstration is to go beyond the internal/external opposition to the profession or even the scientific/managerial alternative and to describe with precision the relationships between different internal actors within the profession.AERES (administrative, scientific, quality experts) and their varying capacity to influence the definition of evaluation procedures.
A representation of the trajectories of forty scientific delegates in HSS and different coordinating delegates in office between 2007 and 2011, thanks to an analysis of multiple correspondences (ACM), allows us to highlight their similarities and dissimilarities. The author identifies three profiles: the scientist (low participation in the governing bodies of the profession), the university manager (responsibilities at the university, research center or discipline level), the political-administrative manager (responsibility at the national or political level). Those who participate in the construction of evaluation procedures are colleagues rather than managers: they are neither experts external to the scientific world nor ex-peers who have permanently distanced themselves from the scientific profession and the professional base. The author describes them intermediate eliteswhich illustrates their position halfway between the ruling elite and grassroots professionals, because they continue to be researchers or teacher-researchers, in practice and in their self-representations.
The author then questions the reasons which push some academics to want to participate in the design of the judgment tools of theAERES (all standards, norms, charters and codes of good conduct for evaluation, from the preparation of the visiting committee to the writing of the evaluation report). After placing the bureaucratization of scientific policies over a longer period of time, two issues seem to explain this movement of rationalization. On the one hand, it responds to critical discourse on existing evaluation bodies, suspected of proximity or even conflicts of interest. On the other hand, the desire to establish valid standards whatever the disciplinary specificities, for the production of homogeneous and comparable evaluation reports refers to a procedural conception of justice which implies that the fairness of the judgment (be) ensured by scrupulous respect for transparent and identical procedures for all (p. 82).
However, despite the desire of the presidents of the agency to implement these standards, the concrete work of the members of theAERES consists rather of a form of DIY, a set of astonishments and internal reflections. This bureaucratization is accompanied by the creation of a conception of good way to evaluate (p. 80), that is to say the ability of evaluators to respect pre-established rules and standards, which is not always well received.
Reinventing and contesting evaluation tools
With two case studies, the author highlights an unequal capacity to challenge the instruments which depends on several factors: characteristics of the protesters, ability to construct a common protest position, solicit legitimate relays, characteristics of the instruments themselves, proximity to the representations and professional practices.
The first case is the mobilization against the ranking of journals by the representativesAERES research in literature (presidents of scholarly associations, sections of the National Committee of Universities, the National Committee of CNRSdirectors of journals in the disciplinary field concerned, certain scientific delegatesAERES). Several sources of this criticism are developed: by relativizing the objectivity and legitimacy of the rankings, these representatives seek to re-politicize the instrument and highlight its multiple biases.; by showing the particularities of the discipline, they oppose rankings and claim their professional autonomy. This discursive work is not enough for the success of the opposition, which is also based on the power relations between the competing institutions of governance of the profession (p. 129). The representatives of the discipline claim their own expertise and manage to mobilize influential relays to contest the establishment of rankings, helped by representative bodies (learned associations such as the French Society of General and Comparative Literature, for example) which have become essential in the making of teaching and research policies (p. 133), if only because some of them suggest the programs of aggregation.
The second case study looks at the establishment in 2011 of an internal consultation lasting several months, in which the author participated and which resulted in the drafting of an evaluation framework defining the conditions of scientificity to which the writings of researchers must subscribe to be considered scientific (p. 143). The survey reports on the process of constructing evaluation standards, the definition of the science being done, the scientific value and finally what “doing research means” (p. 147). An initial work of defining the boundaries of scientific production results in the distinction between science and non-science (popularization, creation, activist commitment). The position adopted consists of not judging the substance but constructing an instrument allowing a formal and procedural evaluation of scientificity, recognizing the heterogeneity of the rules for valorizing academic production in the disciplines of HSS. This makes it possible to assert the position of theAERES outside the scientific controversies specific to each discipline, so as not to favor one practice or conception of science to the detriment of others, more minority or marginal. The author notes, however, that even if the framework does not judge the quality of research results, the formalization of the protocol still has an impact on publication practices and on the laws of scientific recognition (p. 176).
Appropriation of evaluation tools
Finally, the evaluation process and its performative effects are analyzed by comparing two disciplines of HSSliterary studies and geography, using several methods (quantitative study of relationships AERES, interviews, written sources, observation of a visiting committee, that is to say the group of peers who go to the research units and teams to meet the staff and evaluate their work). This comparison is fruitful, because it allows us to show the different appropriation of the instruments: if the literary people discredit the tools ofAERES, geographers make much more systematic use of it. The author explains this in particular by the position of the disciplines in the academic world: geography is represented as a discipline dominated (Bourdieu, 1984, p. 145), while literature is at the opposite pole of the prestige scale in HSS (Bourdieu, 1984). The discipline thus enjoys greater recognition in France, compared to the United States (Duell 2000), with the survey showing no signs of a crisis of legitimacy like that described by geography teacher-researchers. Another explanatory factor is the way in which the reforms are perceived as going in the direction of the interest of the discipline or not: the injunctions to collective work and to international publications go in the direction of the organization of work in geography but weigh on the modes of work in literature.
Conclusion
The author concludes that the establishment ofAERES entered a redistribution within the academic community of the power to denounce the norms that matter (p. 224). The tools of theAERES reform distance scientific practices since they target procedures more than scientific quality.
This work offers elements of understanding of the debates which accompanied the creation of theAERES and will not only be of interest to researchers working on these questions, because it opens up many reflections more widely on important issues for understanding and thinking about the system of research, higher education and scientific work. Although the survey covers the period 2007-2012, the questions it raises are all the more topical given the new framework of the High Council for the Evaluation of Research and Higher Education (HCERESwho took over fromAERES) present in November also sparked a strong reaction from the Assembly of Laboratory Directorates which denounced extreme bureaucratization to the detriment of collegiality and the scientific project of the evaluated units.