University students, like many in society, demonstrate bias against women and particularly women from non-English speaking backgrounds.

That’s the take home message from a new and comprehensive analysis of student experience surveys.

The study examined a large dataset consisting of more than 500,000 student responses collected over 2010 to 2016. It involved more than 3,000 teachers and 2,000 courses across five faculties at the University of New South Wales (UNSW), Sydney.

Most bias in science and business

Interestingly, the bias varies.

In parts of science and business the effects are clear. In the science and business faculties, a male teacher from an English-speaking background was more than twice as likely to get a higher score on a student evaluation than a female teacher from a non-English speaking background.

But in other areas, such as arts and social science, the effects are almost marginal. In engineering, effects were only detected for non-English speakers.

When one looks at the probability of scoring very high ratings, and dissects the categories into genders and cultural background, the results are clear. The disparities occur mostly at the very top end: this is where bias creeps in.

Previously the university had looked at just the average (mean) ratings of teachers of different genders, and found that they are more or less indistinguishable (unpublished data). But this new study goes further and provides information that is not evident in superficial analyses.

Should we abandon student feedback?

Student feedback can be a useful mechanism to understand the varied experiences of students. But student feedback is sometimes used inappropriately in staff performance evaluations, and that’s where the existence of bias creates serious problems.

One can make the case for abandoning student feedback – and many have.

But it’s problematic to turn a deaf ear to the student voice, and that is not what national approaches such as the Quality in Learning and Teaching processes (QILT) are doing.

This is because feedback can often be helpful. It can make things better. In addition, it is often positive. Sometimes the feedback is actually the way students say thanks.

However, sometimes it can be very hurtful and damaging, particularly if it is motivated by prejudice. We have to be aware of that and the barriers it can create.

We know that minority groups already suffer from reduced confidence and visibility, so biased teacher evaluations may exaggerate existing inequities.

What do the numbers mean?

It is very important to be cautious when looking at the raw numbers.

Firstly, let’s consider what the numbers mean. Students are not evaluating teaching and learning in these surveys. They are telling us about their experiences – that’s why we call them MyExperience surveys at UNSW. We resist the idea that they are student evaluations of teaching, as are used in some settings.

Peer review can make contributions to evaluating teaching while assessments can help evaluate learning – however they may not be enough to overcome bias. When considering professional performance at UNSW, we do not exclude the feedback that students provide on their experience, but we look at a basket of indicators.

Secondly, one has to be serious about the biases that emerge, acknowledge them and confront the issues. Most universities pride themselves on being diverse and inclusive, and students support this.

But this study reminds us that we have work to do. Biases exist. The message is strong. You are more likely to score top ratings if you come from the category of white male: that is, if you are from the prevailing establishment.

The influence of history

These results may be surprising given the diversity of the student and staff body at Australian universities.

But our cultural milieu has been historically saturated by white males, and continuing biases exist. The important thing is to be aware of them, and when looking at the numbers to realise that the ratings are provided in the context of a particular society at a particular moment in time.

The scores should not be blindly accepted at face value.

Most universities, including ours, are working on being more inclusive. At UNSW a new Deputy Vice-Chancellor Equity, Diversity and Inclusion – Eileen Baldry – was recently appointed, and we are working hard to combat bias and to introduce new strategies aimed at supporting diversity. For example, the university will introduce new training for members of promotion panels, explaining the biases detected in our new study. By understanding the problem, we can begin to address it.

All staff across all of our universities can benefit from becoming more aware of issues around bias – especially those in powerful positions, such as members of promotion committees.

Reducing bias will have great benefits for society as university students represent a large proportion of future leaders in government and industry.

It is clear that negative stereotypes will contribute to the partiality that exists within our student community. Encouraging more women and cultural minorities at all levels in higher education, in leadership positions and in membership of key committees will help shrink these biases.

Training in values

Training students is challenging, especially at large modern universities such as UNSW, which has a cohort of over 50,000 coming from more than 100 countries. But our study found similar levels of bias in local students, as we did in international students.

In training students we have to remember that we provide knowledge, but also communicate values via our words and our behaviours.

If we are to continue to listen to the student experience, we need to be careful with the results. Rigorous statistical analyses such as this study, can help us recognise bias and work to address it. If our students graduate with less bias than when they entered their degree, we will be contributing to creating a more equitable and inclusive society in the future.

It is not easy to uproot prejudices but the data are clear. We expect people will be on board and be pleased to contribute to moving things in the right direction.

Author Bios: Merlin Crossley is Deputy Vice-Chancellor Academic and Professor of Molecular Biology, Emma Johnston is a Professor and Dean of Science and Yanan Fan is an Associate Professor of Statistics all at UNSW