Major flaws in study on racism in the job market

Last week, an Umno-linked think tank, the Centre for Governance and Political Study, released a study on the issue of racial discrimination in the job market.

The way this study was released is suspect as the organisation chose to release snippets online through their official Twitter account without making the whole research available publicly (they have since made it available).

Upon further scrutiny of the study, we have found multiple methodological flaws, most notably in the way the study presents its seven applicants as being similar in their academic and co-curricular attainment.

This is but the tip of the iceberg in inherent issues with the study’s overarching methodology which will be summarised here.

Methodological flaws

Considering the number of factors the authors were testing (race, appearance, and gender), seven resumes was far too small a number. There needs to be a control resume to ensure that the experiment actually measures what is intended and is not due to some other reason.

For example, as photos of the applicants were included, a control study should have been conducted without photos to see the differences.

The same applies to using a series of race and gender combinations as there is no certainty as to which particular factor was the influential factor in the experiment. Ideally, each condition should be tested with a larger sample size and with multiple conditions to control extenuating factors (e.g. using various photos with different levels of attractiveness).

By using just a handful of resumes to check on multiple factors, there is very little confidence that any one factor can have a significant influence on callback rates.

The claims that the resumes were sent out at random are grossly incorrect. The study stated that they only applied to jobs that were shown by a job portal (this is in no way a form of random sampling as it relies on the job portal’s selection algorithm and excludes other employers).

They only utilised one online job application portal, Jobstreet, and the report did not give the details of the companies the resumes were sent to.

While it is understandable that the report would not want to publish the full name of each of the companies involved, they are still expected to provide demographic details of the companies involved (i.e. size of company, ownership, job scope, type of business, etc).

This lack of transparency with the companies is problematic as it is difficult to tell what the inherent biases or issues are from the list of employers.

There are too many factors that cannot be accounted for, which this report does not indicate, such as the demographic and composition of employers within the study.

A major omission from the methodology is the manner in which the resumes were sent out as there was no clear mention of how companies are selected other than following what was presented by the online job portal.

There is no indication of how often multiple resumes were sent to the same company (i.e. would a company receive all seven resumes?) and whether each of the resumes was sent to a similar set of companies with similar demographics (i.e. were all resumes sent equally to a certain number of SMEs, Bumiputera-owned companies, MNCs, etc?).

This is sorely lacking in the methodology and opens up a lot of doubt about whether the resumes were distributed in a manner that may have skewed the results in a certain direction.

The use of a single model that is then “modified” to create various persons of different ethnicities, while novel, does not do justice to the nuances of ethnicity.

Ethnicity is not merely the colour of a person’s skin. It is a combination of several factors including facial features and even naming conventions which this study seems to be making light of.

The ideal method for this would be to pilot test a series of generic photos with a small group of individuals, rank them based on some form of metric (e.g. attractiveness, trustworthiness) and then use several resumes and photos to represent different races rather than have only one for each gender and race.

If these principles were followed, the number of resumes would be several factors higher than the seven used here.

Many of the justifications for the methods used were just explained without any supporting proof or solid reasoning behind them. There was no reference to other studies that used similar methodologies, nor was there any form of pilot or test studies mentioned that would provide more support for this study.

The authors also failed to control the resumes’ design. Recruiters could have rejected the hijab-wearing candidate simply because it has pink circles which gives out a less professional outlook.

Inconsistency among applicants

This can be seen when we compare the resumes of Gabriel Liew (Chinese male) with that of Thivakar Gunasegaran (Indian male).

Gabriel graduated with Bachelor of Business in banking and finance from Taylor’s University and Queensland University of Technology, Australia, while Thivakar graduated with a Bachelor of Business Accounting from Inti International University, Nilai.

The fact that the two resumes have similar qualifications from two different institutions shows that the study does not have a strong enough control on this variable. This is made worse when one of the qualifications comes from a better-perceived overseas institution.

The study makes the blanket assumption that all private tertiary institutions are equal and fails to control or account for this as a possible factor in the callback rates.

The authors also failed to account for the other differences in the resumes of Nur Sakinah (Malay female, wears hijab) and Zulaikha (Malay female, no hijab).

Zulaikha was presented as a member of the English Debate Club while Nur Sakinah was presented as a member of Persatuan Mahasiswa Islam, which might seem to be a minor difference but would have played a role in the callback rate.

The study reported Zulaikha as having a 40% overall advantage over Nur Sakinah, which the authors attributed to the fact that Zulaikha did not wear a hijab. This conclusion was made without any statistical testing to isolate the so-called “hijab factor” from other potential factors such as the different co-curricular activities.

Why this report is dangerous

The report presents itself as a continuation of a previous academic research paper that is arguably more credible and rigorous than this report.

By doing this, Cent-GPS is attempting to give their report a facade of legitimacy, masquerading it as a genuine academic paper.

The report included four anecdotal statements from an NST piece on workplace discrimination but it does not justify the inclusion of these particular anecdotes in the report.

If the report was genuinely concerned about discrimination, it should have expanded its scope to cover different types of discrimination systematically. The failure to do so points to an attempt to frame discrimination as something imposed by the Chinese.

As allegations involving the Chinese and Malays usually elicit strong emotions, it appears that the inclusion of these anecdotes were meant to stoke racial sentiments.

There is a section in the report that highlights the main assumptions the researchers held with this study:

“We wanted to see if a candidate’s fluency in Mandarin really played a key role in getting a job. As highlighted above, many private sector companies argue that they do many dealings with China-based companies, so they would rather hire a Mandarin-fluent candidate. Is Mandarin fluency really about the skills of the candidate to communicate with clients or is it a smokescreen for employers to pick and choose a certain candidate based on his or her race?”

These statements clearly show that the authors have adopted a biased view of the private sector and an arguably hostile outlook towards virtually all private sector employers.

There is an implicit assumption that all private sector companies are operated or owned by Chinese owners, and that perception has poisoned and influenced every aspect of the report.

However, while the faux-academic nature of the report may be obvious to some, a great majority of the public believe that the report is a bona fide academic paper.

This can be seen from the responses to the report on social media, where many users have taken to citing the report as authoritative evidence on job market discrimination. The situation is exacerbated by the media carrying the report without any critical analysis of the study.

It appears to be that the basis of this report is to reinforce stereotypical views of different ethnic groups and to further racially charged political agendas.

We do agree that there is a problem of racism with employment in Malaysia, as noted in the quoted report by Muhammed Abdul Khalid and Lee Hwok Aun, and that addressing these issues is important for the country as we move forward.

But dishonest studies like this do little to actually improve the current situation. Instead, they pander to societal fears and further worsen inter-race relations.

Sham studies like this need to be critically interrogated and the public needs to be educated better to be able to discern proper social science from faux studies that are meant to warp their world views.

The writers represent DAP’s Parliamentary Research Office. - FMT