Will AI chatbots improve our understanding of masculinity and mental health, or just amplify the existing bias in the literature? A brief comparison of Grok vs ChatGPT

Jan 7

How much do you believe what you read on the internet? I have been concerned for years that people are basing their ideas about men’s mental health on biased information, where the risks of masculinity to mental health are exaggerated and the benefits minimised.

Biased information, usually some version of the deficit model of masculinity, can be found in many places, from the mainstream media, to governments, NGOs, academia and even articles by psychologists. The problem facing anyone searching for information is that the same batch of bad science and poorly-tested theories are recycled uncritically over and over, infecting a multitude of sources that the general public rely on, such as Wikipedia, and sources that academics rely on, such as meta-analyses.

“The problem facing anyone searching for information is that the same batch of bad science and poorly-tested theories are recycled uncritically over and over, infecting a multitude of sources that the general public rely on, such as Wikipedia, and sources that academics rely on, such as meta-analyses.”

With the recent appearance of AI (artificial intelligence) chatbots such as ChatGPT, which offer a quick and easy way of grabbing information on virtually any topic, my concern is that AI will be the prime source of misleading information about men’s mental health, recycling tropes about toxic masculinity and patriarchy that already flood the internet. In order to get some idea of the extent of this problem, I have done a ‘litmus test’ of two of the most popular AI programmes, Grok and ChatGPT. This is not an exhaustive test of course, but importantly is on a topic I have a good knowledge of, having researched and written on the topic for over a decade.

What are Grok and ChatGPT?
Chatbots are a relatively new way of finding information online. Grok is the new kid on the chatbot block, coming after the introduction of ChatGPT, the first big name in the field like (launched in November 2022), and more recently Gemini, and Microsoft’s Copilot. Grok is the brainchild of Elon Musk, owner of X (formerly Twitter).

Grok is uses LLM (large language model) technology that trains the AI by feeding it a vast amount of information from across the internet. Two things that set Grok apart from other chatbots is that it can access content from X, and access content in real time, which makes it useful for understanding breaking news stories and events.

According to a recent article by Android Authority: “the biggest thing setting it [Grok] apart [from other chatbots is] its sarcastic tone and wittiness” e.g. it has a ‘fun mode’ where it can offer cheeky answers. Also, “unlike other chatbots, Grok will respond to more controversial questions that others simply refuse to answer”.

Although Android Authority acknowledge that “reports say that Grok is just as capable and useful as both ChatGPT and Gemini” they claim a weakness of Grok is that it “only has access to data from X […] rather than the wider internet as a whole. […] This means Grok has the potential to perpetuate stereotypes, political agenda, and conspiracies commonly found on social media platforms”. This seems like an unreasonably negative view of X, given the questionable validity of much information on the internet, especially since Android Authority acknowledge that, at the time of writing, they had not tried out Grok yet.

Comparing Grok and ChatGPT re masculinity and mental health
Given the near-ubiquitous nature of negatively biased information about masculinity and mental health, my initial guess was that both chatbots would reflect this status quo. I tried to keep the comparison simple by asking just one question of Grok and ChatGPT: Is masculinity related to mental health? A summary of the answers and my comments are below, and the full answers are listed in the Appendix.

The answer from Grok
The answer from Grok started: “The relationship between masculinity and mental health is complex and multifaceted, influenced by cultural norms, societal expectations, and individual interpretations of what masculinity entails”. Note this implies men’s mental health is not related to biology, even though it is well established that low testosterone is linked to depression, and testosterone is linked to masculinity (e.g. libido, muscle power and fearlessness).

Grok went on to list “some key points from the research and discussions on this topic”, starting with “Negative Perceptions and Mental Health: Research suggests that holding a negative view of masculinity can be linked to lower mental well-being. Studies have shown that men who perceive masculinity as having a negative influence on their behavior tend to experience reduced mental health.” This statement referenced a paper of mine from 2023, and I was impressed that had Grok found it.

“The idea that masculinity is bad for mental health is a form of victim-blaming, and ignores much more obvious causes of men’s mental health problems and suicide, such as the stress related to family breakdown and family courts.”

The next point listed by Grok was disappointing: “Traditional masculine norms like self-reliance, emotional stoicism, and the avoidance of seeking help [potentially lead] to higher rates of untreated mental health problems like depression, substance abuse, or suicide.” This is an oversimplification of a complex issue and overlooks important facts such as that stoicism can be good for mental health. The idea that masculinity is bad for mental health is a form of victim-blaming, and ignores much more obvious causes of men’s mental health problems and suicide, such as the stress related to family breakdown and family courts.

Much better was the next point, that some research indicates that masculinity can be good for mental health. Grok also makes the important point about mental rigidity: “it’s the rigid adherence to detrimental norms that can pose problems”, though this point is undermined by the implying that masculinity is a detrimental norm. People who know about rational emotive behaviour therapy (REBT) know that mental rigidity is a problem when attached to any belief, whether or not the belief is detrimental or useful in its non-rigid form. For example, the idea that ‘it’s better to be good than bad’ can become problematic if it is held too rigidly. In other words, singling out rigidity about masculinity just distracts us from discovering other rigid beliefs that might be more significantly related to a man’s mental health e.g. “I should be happy about family court rulings”.

The next point is that “environments where masculinity is seen as inherently toxic or oppressive, men might feel additional pressure, leading to mental health challenges. Conversely, environments that promote a healthier, balanced view of masculinity might support better mental health outcomes.” It’s a bit unclear whether Grok is saying that men’s perceptions of masculinity are a problem, or the negative narrative about masculinity is the problem, or something else. There could be a good point here about the difficulty men experience living in a culture of misandry , but the point is just not that clear.

The next point is valid regarding the benefits of social interaction, though the inclusion of ‘strict adherence to traditional masculine norms’ being a problem gives the idea that masculine norms are the problem, rather than strict adherence being the problem (see previous point re REBT).

The final point about ‘redefining masculinity’ is potentially a good one, because we very much need for psychologists and sociologists to stop defining masculinity in a negative way. However the implication of the point here is that masculinity needs to change, whereas I would argue that our current narrative about masculinity needs to change. The idea of ‘positive masculinity’ seems like a good one on the surface (I admit to initially being taken in by this one), but is really just a subtle way of saying ‘although masculinity is negative it can be made positive’.

The conclusion is a little disappointing in the way that it subtly suggests that ‘masculinity is ok, if…’, which makes people question masculinity, and approach it with a doubtful mindset: “In summary, masculinity can indeed be related to mental health, both positively and negatively, depending on how it is perceived, enacted, and reinforced within societal, cultural, and personal contexts. The key seems to lie in fostering a version of masculinity that supports emotional expression, social connectedness, and health-seeking behaviors without diminishing the positive aspects of traditional masculinity.”

The answer from ChatGPT
The ChatGPT answer began “Masculinity significantly influences mental health, with both positive and negative effects, shaped by societal and cultural norms”. Like Grok, there was no mention of a potential influence of biology.

Encouragingly, it started by listing “Positive Aspects of Masculinity and Mental Health”, giving as examples Strength, Resilience, Protection and Support. The downside was these were framed as needing to be done by “men who embrace positive masculinity”, implying that masculinity in itself is not positive. The same was true of almost all of the rest of the reply (i.e. masculinity can be positive if it is done correctly) and all of the tropes about ‘traditional masculine norms’ etc were replicated e.g. “Adherence to rigid norms (e.g., “men must always be tough”) can hinder emotional expression and seeking help”. Toxic masculinity was mentioned too, with predictably no alternative explanations for what might cause male behaviour that is labelled ‘toxic masculinity’. Overall this section was a positive spin on a demeaning view of masculinity. Although the APA guidelines were not cited specifically, another APA document supporting the guidelines was cited.

The rest of the answer from ChatGPT was similar to Grok’s (see full answer in Appendix 2), though without the redeeming aspects of Grok’s answer.

“ChatGPT gave the clear impression that masculinity was only ok if it was ‘positive masculinity’. Grok did so too, but to a lesser degree”

Comparison
Both Grok and ChatGPT were similar in that they omitted any sense that masculinity and mental health might be related to biology or evolution. ChatGPT gave the clear impression that masculinity was only ok if it was ‘positive masculinity’. Grok did so too, but to a lesser degree, and this was balanced by including material that was more fully supportive of masculinity. ChatGPT appears to reflect the standard line taken these days by too many men’s organisations, which is a positive gloss on a negative view of masculinity. To its credit, Grok at least provides at least some balance to the questionable parts of its reply. Although I needed to prompt ChatGPT to give me references for the information it provided (“please include the references for your answers”), Grok did so without prompting. Grok also tied in the references more clearly to the information provided, and gave a slightly better range of references, though these included the infamous APA guidelines on therapy with boys and men and similar material.

An important weakness of both Grok and ChatGPT is that they didn’t not offer any evaluation of the evidence provided. I could have probed the chatbots further for such evidence of course, but because many people using chatbots will only quickly dip in and out of a topic, I think it would be far better for the answers to include at least some sense of evaluation at the outset.

Conclusion (Part 1)
When the APA publicised their guidelines for psychological practice with boys and men, many professionals and members of the general public immediately recognised there was something wrong (e.g. the guidelines appeared to endorse questionable assumptions such as hegemonic masculinity and patriarchy theory). You might have thought that the psychology profession would take the criticisms on board, but you would be surprised how much research and practice is still influenced by the APA guidelines, despite their now being more male-centred guidance on therapy with men. Most people will want to know whether an article they are reading subtly endorses ideas such as those in the APA guidelines, so it’s important this information is cited clearly (e.g. in Male Psychology magazine we use hyperlinks). So when you do a search of any kind, whether with a chatbot or not, the results should include the sources presented clearly and accurately.

The qualms expressed by Android Authority - that the information on X would be of lesser quality is - in my opinion, unsupported. It could be that the vast reach and scope of X and its emphasis on free speech make it more likely to include a wider range of information than other chatbots, none of which have direct access to X. For example, even if The Guardian or other left-leaning outlets opt not to post on X themselves, there is no shortage of people who will post Guardian articles for the purposes of discussion. Although Grok is not perfect, it gave at least a two-dimensional picture instead of the one dimension offered by ChatGPT. The fact that the output might have been much worse – imagine a one-dimensional radical feminist view of masculinity – isn’t very redeeming. I think that if we are going to have faith in chatbots, they need to be more like Grok in offering different viewpoints, and better than Grok in offering up-front an evaluation of the different viewpoints.

“I ran the same question in Grok again a few days later […] and was surprised to find a very different answer, and disappointed that it was much more negative about masculinity than the first answer”.

Conclusion (Part 2)
I ran the same question in Grok again a few days later (to get links to references that hadn’t survived the copy & paste into Word), and was surprised to find a very different answer, and disappointed that it was much more negative about masculinity than the first answer (see Appendix 3). I ran the question in ChatGPT again, and got a slightly different answer to the first time, though equally as bad as the first time (see Appendix 4). Based on my admittedly limited experience of chatbots, as described here, it seems my initial fears are confirmed: because the available information online is mainly biased towards the deficit model of masculinity, most samples of this information – or an aggregate of it – will be similarly biased. Should we expect more from large language model technology than ‘garbage in, garbage out’? At the time of writing, this is a disappointing result from the latest that AI can offer, and I hope chatbot developers will seek to remedy this serious flaw. This means people who want to find out about masculinity and mental health should avoid chatbots until such time as they can deliver unbiased information, or at least routinely include alternative viewpoints and evaluations of material in the initial answer.

Appendix 1. Grok’s answer to the question Is masculinity related to mental health? (30th December 2024)
15 posts, 25 web pages [not shown here because they appear as hyperlinked icons, most of which lost their URLs when I copied and pasted to Word].

The relationship between masculinity and mental health is complex and multifaceted, influenced by cultural norms, societal expectations, and individual interpretations of what masculinity entails. Here are some key points from the research and discussions on this topic:

Negative Perceptions and Mental Health: Research suggests that holding a negative view of masculinity can be linked to lower mental well-being. Studies have shown that men who perceive masculinity as having a negative influence on their behavior tend to experience reduced mental health. This includes feelings of isolation, challenges in seeking help, and increased risk of depression and anxiety due to the societal pressure to suppress emotions and appear strong at all times.

Masculine Norms and Help-Seeking Behavior: Traditional masculine norms like self-reliance, emotional stoicism, and the avoidance of seeking help can act as barriers to mental health service utilization. Men, particularly younger men or those from certain demographic groups, might be less likely to seek treatment for mental health issues due to these norms, potentially leading to higher rates of untreated mental health problems like depression, substance abuse, or suicide.

Positive Aspects of Masculinity: On the flip side, some research indicates that positive perceptions of masculinity, including traits like resilience, strength, and assertiveness, can correlate with better mental health outcomes. This suggests that not all aspects of traditional masculinity are harmful; rather, it's the rigid adherence to detrimental norms that can pose problems.

Cultural and Societal Context: The impact of masculinity on mental health can also be influenced by cultural and societal contexts. For instance, in environments where masculinity is seen as inherently toxic or oppressive, men might feel additional pressure, leading to mental health challenges. Conversely, environments that promote a healthier, balanced view of masculinity might support better mental health outcomes.

Social Connectedness and Support: The nature of men's social connections can significantly impact mental health. Men who maintain diverse social networks and do not strictly adhere to traditional masculine norms might be better positioned to seek support when needed, potentially mitigating mental health issues. However, those who prioritize independence and emotional suppression might struggle more with isolation and related mental health problems.

Changing Narratives: There's a growing discourse about redefining masculinity to make it more inclusive and supportive of mental health. Encouraging men to express emotions, seek help, and engage in health-seeking behaviors is seen as a way to promote better mental health among men.

In summary, masculinity can indeed be related to mental health, both positively and negatively, depending on how it is perceived, enacted, and reinforced within societal, cultural, and personal contexts. The key seems to lie in fostering a version of masculinity that supports emotional expression, social connectedness, and health-seeking behaviors without diminishing the positive aspects of traditional masculinity.

Appendix 2. ChatGTP answer to the question Is masculinity related to mental health? (30th December 2024)
Masculinity significantly influences mental health, with both positive and negative effects, shaped by societal and cultural norms.

Positive Aspects of Masculinity and Mental Health

Strength and Resilience: Traits such as resilience and problem-solving can promote mental well-being when expressed healthily.
Protection and Support: Men who embrace positive masculinity may channel traits like protectiveness into supporting their families and communities, fostering connectedness and purpose.

Challenges and Mental Health

Traditional Masculine Norms: Adherence to rigid norms (e.g., "men must always be tough") can hinder emotional expression and seeking help, contributing to mental health issues like depression and anxiety.

ADAA

Social Expectations: Men often face societal pressure to conform to ideals of dominance and independence, leading to:

Suppressed emotions.
Reluctance to seek professional help.
Increased stress and feelings of inadequacy.

Toxic Masculinity: Overemphasis on dominance and emotional suppression can lead to maladaptive behaviors, interpersonal conflict, and poor mental health outcomes.

ADAA

Men's Mental Health Statistics

Men are less likely than women to seek help or receive treatment for mental health issues.

American Psychological Association

In the United States, men die by suicide at a rate four times higher than women.

PubMed Central

Over 6 million men suffer from depression per year, but male depression often goes underdiagnosed.

ADAA

Promoting Healthy Masculinity

Encouraging Emotional Expression: Normalizing that men can express a range of emotions without judgment can reduce stigma and support mental health.
Redefining Strength: Strength can include asking for help, being vulnerable, and building supportive relationships.
Education and Awareness: Raising awareness about the impact of gender norms on mental health can foster healthier attitudes and behaviors.
Accessible Support Systems: Creating spaces where men feel safe discussing their mental health without fear of judgment is crucial.

Understanding masculinity and its complex relationship with mental health is key to creating supportive environments that promote well-being for all individuals.

Recent Discussions on Masculinity and Mental Health

The Times. Why is it that men still find it hard to ask for help? 38 days ago

Health.com How Toxic Masculinity Affects People and What Can Be Done About It, 172 days ago

Time. The Precarious Online Lives of Young Men. 138 days ago

Appendix 3. Grok’s second answer to the same question (2nd January 2025)
15 posts, 25 web pages [not shown here because they appear as hyperlinked icons, most of which lost their URLs when I copied and pasted to Word].

Yes, masculinity is indeed related to mental health, and this relationship is complex and multifaceted. Here's how:

Gender Norms and Emotional Expression: Traditional masculine norms often discourage emotional expression, encouraging men to be stoic, independent, and to suppress feelings like sadness or anxiety. This can lead to mental health issues because bottling up emotions can contribute to stress, depression, and anxiety. Research indicates that when men adhere to rigid masculine norms, they might experience higher levels of negative risk-taking and inappropriate aggression, which can be detrimental to their mental health.

Help-Seeking Behavior: There's a significant body of evidence suggesting that men are less likely to seek mental health services due to these norms. The pressure to appear strong and self-reliant can make men reluctant to admit when they need help, leading to untreated mental health conditions. This reluctance is often linked to societal expectations of masculinity, where seeking help might be seen as a sign of weakness.

Social Connections: Masculinity can also influence social connectedness. While some men might have strong support networks, others might struggle due to norms that emphasize independence over interdependence. This can lead to social isolation, which is a risk factor for mental health issues like depression. There's a diversity in how men engage with social support, suggesting that not all masculine practices are detrimental, but certain patterns can indeed impact mental well-being negatively.

Positive Aspects of Masculinity: However, masculinity isn't always harmful to mental health. Some studies suggest that certain masculine traits, like self-confidence and a sense of purpose from traditional masculine roles, can be protective. A positive view of masculinity might contribute to better mental health outcomes, as it can encourage resilience and provide a sense of identity and belonging.

Cultural and Societal Impact: The societal and cultural context also shapes how masculinity affects mental health. In some cultures, there's a growing recognition of the need to redefine masculinity to promote healthier behaviors, including emotional openness and seeking help when needed. This shift is seen as crucial in addressing the mental health crisis among men, particularly in combating high rates of suicide, substance abuse, and other mental health challenges.

Intersectionality: The intersection of masculinity with other identity factors like race, socioeconomic status, and sexual orientation can further complicate the relationship with mental health. For instance, men from marginalized groups might face compounded stigma, reducing their likelihood of seeking mental health support due to both masculine and racial or economic pressures.

In summary, masculinity's relationship with mental health is not straightforward; it involves both risks and benefits. The traditional aspects of masculinity can indeed lead to poorer mental health outcomes if they promote emotional suppression and isolation, but evolving norms that encourage emotional literacy and support-seeking might mitigate these risks. The challenge lies in balancing the beneficial aspects of traditional masculinity with modern health practices that encourage emotional expression and mental health care.

Appendix 4. ChatGPT’s second answer to the same question (2nd January 2025). [References not requested this time].

Yes, masculinity is related to mental health in several ways. The concept of masculinity, often shaped by cultural, social, and personal factors, can influence mental health outcomes, help-seeking behaviors, and coping mechanisms. Here’s a closer look at this relationship:

1. Traditional Masculine Norms

Pressure to Conform: Traditional notions of masculinity often emphasize traits such as emotional stoicism, self-reliance, and toughness. This can discourage men from expressing vulnerability or seeking help for mental health challenges.
Emotional Suppression: Suppressing emotions like sadness or anxiety to appear "strong" can lead to issues like increased stress, depression, and difficulty forming supportive relationships.

2. Mental Health Stigma

Men are often less likely than women to seek professional help for mental health concerns due to societal stigma. This can delay diagnosis and treatment, exacerbating mental health problems.
Fear of judgment or being perceived as weak can prevent men from sharing their struggles even with close friends or family.

3. Risk Behaviors and Coping Strategies

Men who adhere to traditional masculine norms may turn to unhealthy coping mechanisms, such as substance use or risky behaviors, as a way to manage emotional distress.
These behaviors can lead to additional mental health challenges, such as addiction or feelings of guilt and shame.

4. Impact on Suicide Rates

Globally, men are more likely to die by suicide than women, often attributed to factors like reluctance to seek help, emotional isolation, and the use of more lethal means.
The societal expectation to "deal with problems alone" can make men feel trapped or hopeless.

5. Evolving Masculine Ideals

Positive Masculinity: Emerging discussions about masculinity emphasize traits such as emotional intelligence, compassion, and vulnerability, which can promote better mental health and stronger interpersonal relationships.
Encouraging open conversations and redefining masculinity to include seeking support and prioritizing mental well-being can reduce stigma and improve outcomes.

6. Intersectionality and Masculinity

Masculinity intersects with other identities, such as race, class, and sexual orientation, influencing mental health in complex ways. For instance, cultural expectations around masculinity might differ, further affecting mental health experiences and access to care.

Promoting Healthy Masculinity and Mental Health

Education and Awareness: Programs that challenge traditional norms and encourage emotional openness can foster resilience and reduce stigma.
Role Models: Public figures and community leaders demonstrating vulnerability and seeking help can normalize these behaviors.
Accessible Services: Tailoring mental health services to be more appealing and approachable for men can improve engagement and outcomes.

Addressing the relationship between masculinity and mental health requires a combination of cultural change, education, and individual empowerment. Recognizing and challenging harmful norms can help create a more supportive environment for everyone.

Recommended reading and resources

Seager, M., Barry, J.A. (2019). Positive Masculinity: Including Masculinity as a Valued Aspect of Humanity. In: Barry, J.A., Kingerlee, R., Seager, M., Sullivan, L. (eds) The Palgrave Handbook of Male Psychology and Mental Health. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-04384-1_6

Barry, John; Walker, Rob; Liddon, Louise; & Seager, Martin. (2020). Reactions to contemporary narratives about masculinity: A pilot study. Psychreg Journal of Psychology, 4(2), 8–21. https://doi.org/10.5281/zenodo.3871217

Liddon, L., & Barry, J. (2021). Perspectives in male psychology: An introduction. John Wiley & Sons. ISBN: 978-1-119-68535-7

Barry J. (2023). The belief that masculinity has a negative influence on one's behavior is related to reduced mental well-being. Int J Health Sci (Qassim). PMID: 37416841

Centre for Male Psychology (2023). Introduction to Male Psychology and Mental Health. (Online course, approved by the British Psychological Society (BPS) for the purposes of continuing professional development (CPD)).

Scroll down to join the discussion

Disclaimer: This article is for information purposes only and is not a substitute for therapy, legal advice, or other professional opinion. Never disregard such advice because of this article or anything else you have read from the Centre for Male Psychology. The views expressed here do not necessarily reflect those of, or are endorsed by, The Centre for Male Psychology, and we cannot be held responsible for these views. Read our full disclaimer here.