Photo: Guy Ackermans The career structure at the university, tenure track, is being overhauled. The current system relies too heavily on quantitative assessment criteria such as the number of publications or PhD students. WUR is looking for a better assessment system that evaluates the quality of its scientists.
‘When I started,’ says the Wageningen tenure track professor Sylvia Brugman, ‘promotion from assistant professor to associate professor was the biggest test on tenure track. At that point it is ‘up or else’: either you get a permanent post in the chair group or you have to look for another job.’
Scoring points
In the tenure track system Brugman is referring to, credits points play a decisive role. Researchers have to score points with publications in journals. The more prestigious the journal – in other words, the higher its impact factor -, the more points you score. But what are we actually measuring that way? According to a group of American cell biologists who drew up the San Francisco Declaration on Research Assessment (DORA) in 2012, the impact factor does not set out to assess the academic quality of an article. That’s because of a flaw in the measurement: by giving journals scores, review articles end up being given the same importance as articles with new insights and breakthroughs. What is more, journals can manipulate their impact factors. It would be better to evaluate the quality and impact of individual articles rather than of the journal.
Distorted picture
Besides this publication score, Tenure track staff are evaluated on their teaching. They are expect to teach 560 hours per year. They need to get high ratings in student course evaluations – at least 3.7 on a scale of 5. But a student evaluation is not a recognized method of measuring teaching quality. Typically, not more than 30 per cent of the students fill in the survey, and the results give a distorted picture because only discontented and very enthusiastic students give their feedback. In spite of that, tenure track staff who score 3 or 3.5 are often told their teaching is not good enough.
PhD factory
But the main focus of Wageningen scientists’ criticism is on the large number of PhD students they have to attract and supervise in order to rise up the ranks. Tenure track staff must supervise a minimum of five or six PhD students to get promoted to associate professor status, and at least 10 to obtain a personal professorship. So everyone on tenure track needs to have their own little ‘PhD factory’ producing a constant stream of publications and citations in top journals.
The falling success rate for research proposals submitted to financiers such as the Netherlands Organization for Scientific Research NWO and the EU is making it harder and harder for tenure track staff to recruit the required number of PhD students. And the quest for funding is becoming more and more time-consuming and stressful. By way of illustration, Brugman talks about her quest for funding for a research proposal. She first submitted it to a Wageningen Graduate School. After being rejected – six months later – she submitted a revised proposal to the NWO. After another rejection – nine months later – she submitted a further revised proposal to a different NWO fund. This time she was successful. ‘The whole process took three years. By that time, your innovative research proposal has already been assessed by three groups working in your field. That detracts somewhat from its innovative character.’
Flaw
Tenure track staff are also aware that the Wageningen career structure has been flawed from the start in 2010. If all tenure track candidates successfully progress through the system, in 10 years’ time the university will be made up entirely of personal professors and PhD students, its critics predicted 10 years ago. That is not what a university wants. In the ideal chair group, there is also a role for assistant and associate professors who stay at those levels.
The Executive Board solved that problem in 2016. Tenure track staff were no longer obliged to seek promotion to a personal chair, but could stop at the associate professor level. The Board also sharpened the criteria for becoming a personal professor. This angered tenure track staff who were aiming at becoming personal professors. But now that the labour market for scientists is tight, this change is starting to work against WUR. Talented academics can see that other universities require fewer credit points to get promotion to personal professor. That might prompt them to apply elsewhere, fear Wageningen’s chair-holding professors.
Work pressure
Finally, there is another development that is putting pressure on tenure track staff. The growing student population in Wageningen is pushing up the work pressure for them too. They spend more time on course administration, such as answering emails from students about the timetable, classroom or exam. Tenure track staff who take the time for all this get told by their evaluation committee that they are doing too much teaching and not enough research. ‘All together, it’s too much of a good thing,’ says tenure track professor Kevin Matson. ‘You can’t supervise 10 PhD students and write five top publication and deliver excellent teaching. High scores are currently the ticket to promotion. But what is the point of having 10 PhD students if you haven’t got the time to supervise them?’
Not quantitative but qualitative
How do we arrive at a better system of evaluation? According to the American DORA declaration, articles that are evaluated by colleagues are still the most important indicator of research quality. But those articles should be assessed on their own merits rather than on the ranking of the journal in which they are published. Furthermore, the researcher’s peers should also assess other forms of academic output, such as data sets.
The DORA declaration also states that universities should broaden their impact assessment. Citations are a sign of impact, but so is the influence of research on policy and practice. This calls for qualitative assessment by experts who can evaluate the research in question, rather than for figures. WUR has signed this DORA declaration, so it wants to act on in.
A second group feels that the tenure track system places too much emphasis on individual research quality, whereas a good academic should also contribute to broader research and education programmes. Partly as a result of this, some argue for evaluating groups from now on, rather than individuals. The idea behind this is: individuals cannot excel at everything, but a group can excel at both education and research. A collective assessment of this kind is in the spirit of WUR’s new strategic plan, entitled ‘Finding answers together’.
In practice
How can WUR put this into practice? ‘WUR already has education evaluations, portfolios and committees that assess the scientists more broadly, so all the elements for a qualitative assessment are already in place,’ says tenure track professor Matson. WUR could also consult the University of Ghent, which radically adapted its evaluation policy in 2018. The Belgian university no longer evaluates its scientists using quantitative indicators such as the number of publications in top journals and the number of PhD students, but has opted for a qualitative evaluation. Now, every five years, the researchers write an account of the role they wish to play in their chair group and how they believe their research and teaching contributes to the university. After five years they themselves evaluate what they are proudest of – whether a publication, a course or an application of their knowledge – after which a five-person committee assesses their development.
The biggest advantage of this new evaluation system to date, says the Ghent rector Rik van de Walle, is that it generates much less administration. He does note a few teething troubles, though. ‘I notice that people who have to assess our full professors struggle a bit with the role. They used to be able to hide behind impersonal forms. If the professor got a low score for publications or PhD students, it was “the system” that said so. Now you have to argue your case for saying the professor’s performance is below par, and that is much harder. It is a culture change, in which the evaluator is also a coach.’
Watch this space
Van de Walle’s advice to other universities is to enter into dialogue with their scientists to discuss alternative for the score-based tenure track criteria. And that is exactly what WUR plans to do this year. The Executive Board has appointed a committee that will explore the options. That committee, says its chair Arnold Bregt, will take plenty of time and will be holding meetings about tenure track over the coming months. Watch this space.
Higher up: how it works
At present, researchers on tenure track can work their way up in five steps from assistant professor (2 and 1) and associate professor (2 and 1) to personal professor. Promotion occurs on the basis of credits or points amassed with research (i.e. publications), education (i.e. student evaluations) and the supervision of PhDs and postdocs.
To get promoted to the level of associate professor, Wageningen natural scientists have to accrue an average of 20 publication points per year. You get six points for a publication in a Q1 journal – one of a group of journals that are among the best 25 per cent in their field, based on impact. An article in a Q2 journal – in the second quartile – earns you three points. So an article in Nature gets you 20 points. The further you get on tenure track, the more articles you need to publish in Q1 journals.
The professors are also expected to teach a minimum of 560 hours per year. As supervisors they have to accumulate 24 points, and one way of doing that is to be a PhD student’s co-supervisor for three years (10 points a year). The criteria become more stringent with each rung on the career ladder, and they vary per science group. Academics up for promotion are assessed by the Broad Assessment Committee (BAC), which considers not only at the required credits but also the person’s qualities in the areas of acquisition and collaboration.
New measurement
The way academics are assessed is too one-sidedly based on research production, with very little attention to any other talents. A new way of evaluating and recognizing achievements has been designed to change that. This is the key objective of ‘Room for everyone’s talent’, a joint policy document from the Dutch Universities Association (VSNU) and the country’s big research funding bodies NWO, KNAW, ZonMW and NFU.
A new instrument will be used to measure scientists’ performance, with a new emphasis on quality rather than quantity. Besides research, quality can be reflected in education, leadership, social impact and – for medical schools – patient care. You can create impact by, say, making research results accessible to society or involving a wider public in science through citizen science. Leadership can be shown in the supervision of students or PhD researchers. Collaboration and the contribution made to the group are central, and not just how the individual functions. Research and education are still important, but you can now score for standing out in other areas too.
The VSNU is going to develop a new assessment system this year, to come into effect from 2021. Each university will use this to draw up its own assessment criteria. The idea behind this new approach to evaluation is to increase the diversity of the people and the career trajectories in academia, says the VSNU. The shift from quantity to quality should also reduce the work pressure. Today’s world, with its complex social and scientific issues, cries out for multidisciplinary collaboration among scientists with a range of different capacities, says the policy paper.
