Is our attitude to men based on substandard research?
Much of our attitude towards men comes from high profile feminist and Gender Studies scholars in academia. There is a strong association between the theorizing amongst such academics and the ideologies expressed in public debate. In fact, it is often the same individuals who write op-eds, scholarly papers, and pamphlets, and engage in political activism. It is only the type of platform that varies. Much of the ideas about male privilege, quotas, “rape culture”, and the patriarchy, for example, began in academic departments. But what is the scientific quality of publications from the world of gender studies?
We were compelled to investigate the quality of gender studies publications after some alluring lectures given at Umeå University, arguing that Gender Studies (GS) outperforms the old-fashioned, bigoted, and boring positivist methods. GS, it was argued, is an alternative and superior kind of science.
“we compiled a database of all GS [Gender Studies] publications we could find in a certain time frame, written by scholars active in Sweden.”
This aroused our interest, so we compiled a database of all GS publications we could find in a certain time frame, written by scholars active in Sweden. This Swedish Gender Studies List (SGSL) contains 12,414 cases, and has to date been used in three studies, all published in the journal Scientometrics (Söderlund & Madison, 2015; 2017; Madison & Söderlund, 2018). As they are already reported in detail, we provide only a brief summary of selected results here.
“GS articles are substantially more often book chapters, dissertations, and conference contributions, and less often peer-reviewed journal articles… [and] significantly higher proportions of biased and normative statements”
The 2015 study found that the annual growth rate of GS publications was greater than for research in general. In the period 2000 to 2010, GS publications from Sweden increased ~12% per year and ~7% internationally, while other research both in Sweden and internationally grew ~3-7%.
GS articles are substantially more often book chapters, dissertations, and conference contributions, and less often peer-reviewed journal articles (~20% of GS and 70% on non-GS publications).
Overall, non-GS articles had 2-3 times more citations than GS articles, and ~90% have 3 citations or more compared to only ~28% of GS articles.
50% of GS journals were not even indexed in Thomson’s Journal Citation Reports, and the bulk of the remaining journals’ had an impact factor of less than half that of the average for non-GS journals (~1.0, as compared to ~2.0).
The 2016 and 2018 studies were based on ~2,800 statements culled from 36 journal articles with more and less gender perspective, and found significantly higher proportions of biased and normative statements in GS articles, and lower proportions of statements about biology/genetics and individual/group differences, than in non-GS articles. Consistent with this, non-GS articles had significantly lower proportions of statements about environment/culture and societal institutions.
The 2018 study found that GS texts were more abstract, less empirical, and focussed to a greater extent on societal factors. The proportion of statement of fact was 82% for GS articles but 48% for non-GS articles. Correlations were mentioned as a relationship between variables in 10% of GS statements and 37% of non-GS statements, but there was no difference in the proportion of causal relations (~5% for both GS and non-SG).
GS articles had less mention of limitations and earlier theories, results, and research in general, and had less support for their statements in terms of arguments or references.
In conclusion, GS articles appear to focus on communicating examples of different experiences and viewpoints of certain groups of people, rather than comprehensive models of the real world.
It is quite unusual that scholars comment on each other’s articles in print, but our 2015 article was challenged by ten pages listing alleged errors, asserting that it “falls well short of adequate good practice”, that we “distort” the conceptual framework, are “unreflecting”, and “lack…understanding” and ”…knowledge”. We were, however, unable to find any argument as to how the many but relatively minor concerns listed might, separately or taken together, challenge the main result that more gender perspective was associated with lower scientific quality. The implication was not lost upon us that only gender studies scholars can be trusted to criticize gender studies publications..
“it is interesting and potentially illuminating that GS [Gender Studies] scholars find it worthwhile to launch arrows that so clearly miss the mark”.
From an epistemological perspective it is interesting and potentially illuminating that GS scholars find it worthwhile to launch arrows that so clearly miss the mark. The data are in. We are just saying what they look like. A comment to our 2018 article was just published, again with a long list of grievances about definitions of concepts and our lack of competence. In essence, however, both comments argue that the differences we document are expected, and are therefore not valid critique. Maybe so, but that does not refute our argument that this field could achieve a greater impact if those differences were reduced.
Surely, it stands to reason that any communication would be more useful as a source of knowledge and guide for action and change if its scientific quality were higher? Is it not obvious that such quality is associated with more, rather than less, data, reliability, validity, non-bias, self-criticism, and most other quality indicators addressed in our studies, the very same indicators that scientific journals in general apply as criteria for peer-review? (Although perhaps not all journals).
We welcome the debate, and look forward to hear representatives of GS develop their arguments for why these criteria are undesirable for, or do not apply to, their field.
This article was first published on the Male Psychology Network website in January 2018
Scroll down to view comments
Disclaimer: This article is for information purposes only and is not a substitute for therapy, legal advice, or other professional opinion. Never disregard such advice because of this article or anything else you have read from the Centre for Male Psychology. The views expressed here do not necessarily reflect those of, or are endorsed by, The Centre for Male Psychology, and we cannot be held responsible for these views. Read our full disclaimer here.
Like our articles?
Click here to subscribe to our FREE newsletter and be first
to hear about news, events, and publications.
.
Guy Madison is a professor of Psychology at Umeå University. He has authored some 150 scientific papers across diverse fields of study (human timing modelling, music psychology, intelligence, personality, and individual differences, evolutionary perspective and behavioural genetic methods). He mainly teaches behavioural genetics and advanced level scientific methodology.
Therese Söderlund has a Master’s degrees in Psychology and in Worklife and Health, and Bachelor’s degrees in Swedish and English. She has been employed as a researcher and project assistant at the Department of Psychology at Umeå University since 2009.