The Jason Arday affair must not spell the end for diversity. Here’s how – and why – we should defend it | Joseph Harker

This has not been a good fortnight for the concept of diversity and inclusion. The fallout from the Jason Arday affair has been an all-out attack, by the rightwing media and political classes, on institutional efforts to reduce inequality.

Arday was the Black academic who, in a fanfare of publicity, had been appointed a professor at Cambridge University in 2023. At the time, British institutions were still struggling to address the widespread issues of systemic racism thrown up by the murder of George Floyd three years earlier.

Last month Arday faced allegations of plagiarism over his PhD thesis, and amid massive scrutiny he resigned last week. Cambridge University has now announced an independent inquiry into its appointment procedures, but many academics already believe that Arday was given the job more for his remarkable backstory than for his academic ability. Arday was, they felt, not at the level you would expect for a professorial chair at such an elite institution. But he was the man who said he couldn’t talk till he was 11, and couldn’t read till he was 18; who had run 30 marathons in 35 days, the last third of them with a leg fracture. What an amazing story. Arday gathered headlines and media appearances. He became a walking advert for how an institution with such a poor record on diversity had progressed so far. Or had it?

I’ve been promoting diversity and inclusion for more than 25 years, and one of the stock responses I often hear is: “I’d love our organisation to be more diverse, but we don’t want to lower standards.” It trips off the tongue so smoothly, always accompanied by an apologetic faux-empathetic shrug.

Yet the response completely misunderstands what diversity and inclusion is about. If it meant lowering standards I’d never waste my time pushing it. Who could possibly believe in making their organisation worse? D&I is not about making a moral point that you believe in equality (though you should); it’s not about performing an act of charity because you feel sorry for poor, excluded minorities (though you might); it’s not even about a business case in the expectation of making more money (though you also might).

The point of D&I is that you bring people into your organisation with different life experiences and perspectives; you value them and listen to them (that’s why it’s called “inclusion”); and the outcome is that you make more informed decisions, so that your organisation better serves its customers. This may or may not translate into profits, but it definitely makes you better, and able to spot changing trends and changing patterns of behaviour more quickly.

Opening up your hiring to more diverse communities leads to more competition and therefore higher standards.

This is also why it’s important to make the distinction between positive action and positive discrimination.

Positive action is inclusive and offers training or short-term experiences to underrepresented groups, helping to level the playing field before they apply for permanent jobs. Positive discrimination (commonly referred to as quotas), excludes people from job applications, reduces the numbers who are able to apply, and therefore lowers standards.

Positive discrimination has always been illegal in the UK; its US equivalent, “affirmative action” was legal but provoked a huge backlash and is now in effect outlawed. Positive discrimination also leads to minorities in any organisation having doubt cast on their position, that somehow they’re second rate, the “diversity hire”.

The backlash over the Arday appointment has cast this shadow once again over a whole swathe of Black academics. Many of them are now furious at what Cambridge did. If the university had wanted to open the door for a Black professor, there are dozens of academics at the right level. The difference is that they wouldn’t have been such a PR hit. The message they’ve taken from Cambridge is that, if you’re Black, you only get a top job if you have a superhuman backstory. Level playing field? Really?

If Cambridge really were an inclusive institution it wouldn’t have been looking for the “quick fix” that the Arday appointment gave it: a superficial symbol that everything here is OK. Instead it would have been examining its recruitment processes, the retention and promotion rates of its minority academics, to see where the problems were and what could be done. Cambridge would ask: how can we identify true potential and what kind of support can we offer people from underrepresented groups to ensure they thrive in the institution? (This applies just as much to women, working-class people and those with disabilities.)

The past 18 months, since Donald Trump regained the US presidency, have seen huge political efforts to turn the clock back on equality and racial justice. Many companies have stood firm; but some have caved in and ditched their policies. The difference is that some understand why diversity and inclusion matters, and some don’t: there is ample evidence that diverse organisations perform better.

So what should be the lesson for progressives who believe in equality? If, after the past few weeks’ headlines, any institution feels under renewed pressure to ditch its diversity policies, the correct response is: “But we don’t want to lower our standards.”

• Joseph Harker is the Guardian’s senior editor, diversity and development

• Do you have an opinion on the issues raised in this article? If you would like to submit a response of up to 300 words by email to be considered for publication in our letters section, please click here.