Brain activity during emotion recognition from eye expressions

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription Access

Abstract

In the current study, we conducted a psychophysiological approbation of the previously developed Russian-language version of the Reading the Mind in the Eyes Test (RMET) using functional magnetic resonance imaging (fMRI). In the statistical analysis, we included 40 subjects (13 males/27 females, mean age 22.5 ± 3.2 years). As a result, reproducible RMET-associated increases in the level of local neuronal activity compared to a control condition (age recognition) were found for a variety of brain structures that previously have shown to be involved in providing processes for affective state recognition from eye expression, including the inferior frontal gyrus, the supplementary motor cortex of both hemispheres, left precentral gyrus, and right middle/upper temporal gyrus. At the same time, the current work demonstrated new data about cerebellum and thalamus involvement in affective state recognition compared with age recognition during the RMET task, which was reported in isolated studies. Additionally, new effects of changes in local neuronal activity during the recognition of affective states have been observed. These findings clarify the role of the medial prefrontal cortex, precuneus, left angular gyrus, and left middle/upper temporal gyrus in understanding the internal emotional and mental states of others. As a result of the fMRI study, the new Russian-language version of the RMET task was successfully tested. This approbation provided additional data that enhances our understanding of the brain structures involved in the affective aspects of mentalization-related processes.

Full Text

Restricted Access

About the authors

M. A. Zheltyakova

Bekhtereva Institute of the Human Brain of the Russian Academy of Sciences

Email: korotkov@ihb.spb.ru
Russian Federation, Saint Petersburg

A. D. Myznikov

Bekhtereva Institute of the Human Brain of the Russian Academy of Sciences

Email: korotkov@ihb.spb.ru
Russian Federation, Saint Petersburg

A. D. Korotkov

Bekhtereva Institute of the Human Brain of the Russian Academy of Sciences

Author for correspondence.
Email: korotkov@ihb.spb.ru
Russian Federation, Saint Petersburg

I. S. Knyazeva

Bekhtereva Institute of the Human Brain of the Russian Academy of Sciences

Email: korotkov@ihb.spb.ru
Russian Federation, Saint Petersburg

R. S. Masharipov

Bekhtereva Institute of the Human Brain of the Russian Academy of Sciences

Email: korotkov@ihb.spb.ru
Russian Federation, Saint Petersburg

D. V. Cherednichenko

Bekhtereva Institute of the Human Brain of the Russian Academy of Sciences

Email: korotkov@ihb.spb.ru
Russian Federation, Saint Petersburg

M. V. Kireev

Bekhtereva Institute of the Human Brain of the Russian Academy of Sciences; Saint Petersburg State University

Email: korotkov@ihb.spb.ru
Russian Federation, Saint Petersburg; Saint Petersburg

References

  1. Александров Ю.И. Психофизиология, 3rd ed. Питер, СПб. 2007.
  2. Бурсов К.Г., Желтякова М.А., Мызников А.Д., Морошкина Н.В., Князева И.С., Машарипов Р.С., Коротков А.Д., Киреев М.В. Валидация русскоязычного аналога RMET: лингво-психологическое исследование для создания инструмента для нейровизуализационных исследований. Десятая Международная Конференция По Когнитивной Науке: Тезисы Докладов 2024. Десятая международная конференция по когнитивной науке, Пятигорск: 59–60.
  3. Румянцева Е.Е. Методика оценки психического состояния другого по выражению глаз. Психиатрия 2013. 59 (3): 30–36.
  4. Сахибалиева А.А., Прошутинский Е.О., Мершина Е.А., Печенкова Е.В. Чтение психического по глазам и голосу: разработка методик для фМРТ на русском языке, in: ПСИХОЛОГИЯ ПОЗНАНИЯ Материалы Конференции. Всероссийская Научная Конференция Памяти Дж. С. Брунера 2024. 336-340.
  5. Adams R.B., Rule N.O., Franklin R.G., Wang E., Stevenson M.T., Yoshikawa S., Nomura M., Sato W., Kveraga K., Ambady N. Cross-cultural reading the mind in the eyes: an fMRI investigation. Journal of cognitive neuroscience. 2010. 22(1): 97–108. https://doi.org/10.1162/JOCN.2009.21187
  6. Altschuler M., Sideridis G., Kala S., Warshawsky M., Gilbert R., Carroll D., Burger-Caplan R., Faja S. Measuring Individual Differences in Cognitive, Affective, and Spontaneous Theory of Mind Among School-Aged Children with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders. 2018. 48(11): 3945–3957. https://doi.org/10.1007/S10803-018-3663-1/FIGURES/3
  7. Andrews T.J., Ewbank M.P. Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe. Neuroimage. 2004. 23(3): 905–913. https://doi.org/10.1016/j.neuroimage.2004.07.060
  8. Arioli M., Cattaneo Z., Ricciardi E., Canessa N. Overlapping and specific neural correlates for empathizing, affective mentalizing, and cognitive mentalizing: A coordinate-based meta-analytic study. Human Brain Mapping. 2021. 42(14): 4777–4804. https://doi.org/10.1002/HBM.25570
  9. Balgova E., Diveica V., Jackson R.L., Binney R.J. Overlapping neural correlates underpin theory of mind and semantic cognition: Evidence from a meta-analysis of 344 functional neuroimaging studies. Neuropsychologia. 2024. 200: 108904. https://doi.org/10.1016/J.NEUROPSYCHOLOGIA.2024.108904
  10. Baron-Cohen S., Ring H.A., Wheelwright S., Bullmore E.T., Brammer M.J., Simmons A., Williams S.C.R. Social intelligence in the normal and autistic brain: An fMRI study. European Journal of Neuroscience. 1999. 11(6): 1891–1898. https://doi.org/10.1046/J.1460-9568.1999.00621.X
  11. Baron-Cohen Simon, Wheelwright S., Hill J., Raste Y., Plumb I. The ’’Reading the Mind in the Eyes’’ Test Revised Version: A Study with Normal Adults, and Adults with Asperger Syndrome or High-functioning Autism. J. Child Psychol. Psychiat. 2001. 42(2): 241–251.
  12. Bodden M.E., Kübler D., Knake S., Menzler K., Heverhagen J.T., Sommer J., Kalbe E., Krach S., Dodel R. Comparing the neural correlates of affective and cognitive theory ofmind using fMRI: Involvement of the basal ganglia in affective theory ofmind. Advances in Cognitive Psychology. 2013. 9(1): 32. https://doi.org/10.2478/V10053-008-0129-6
  13. Bora E. A meta-analysis of theory of mind and “mentalization” in borderline personality disorder: a true neuro-social-cognitive or meta-social-cognitive impairment? Psychological medicine. 2021. 51(15): 2541–2551. https://doi.org/10.1017/S0033291721003718
  14. Brothers L., Ring B. A Neuroethological Framework for the Representation of Minds. Journal of Cognitive Neuroscience. 1992. 4(2): 107–118.
  15. https://doi.org/10.1162/JOCN.1992.4.2.107
  16. Castelli I., Baglio F., Blasi V., Alberoni M., Falini A., Liverta-Sempio O., Nemni R., Marchetti A. Effects of aging on mindreading ability through the eyes: An fMRI study. Neuropsychologia. 2010a. 48(9): 2586–2594. https://doi.org/10.1016/J.NEUROPSYCHOLOGIA.2010.05.005
  17. Coad B.M., Postans M., Hodgetts C.J., Muhlert N., Graham K.S., Lawrence A.D. Structural connections support emotional connections: Uncinate Fasciculus microstructure is related to the ability to decode facial emotion expressions. Neuropsychologia. 2020. 145: 106562. https://doi.org/10.1016/j.neuropsychologia.2017.11.006
  18. Deng F., Bueber M.A., Cao Y., Tang J., Bai X., Cho Y., Lee J., Lin Z., Yang Q., Keshavan M.S., Stone W.S., Qian M., Yang L.H., Phillips M.R. Assessing social cognition in patients with schizophrenia and healthy controls using the reading the mind in the eyes test (RMET): a systematic review and meta-regression. Psychological Medicine. 2024. 54(5): 847–873. https://doi.org/10.1017/S0033291723003501
  19. Diveica V., Koldewyn K., Binney R.J. Establishing a role of the semantic control network in social cognitive processing: A meta-analysis of functional neuroimaging studies. NeuroImage. 2021. 245: 118702. https://doi.org/10.1016/J.NEUROIMAGE.2021.118702
  20. Engell A.D., Haxby J.V. Facial expression and gaze-direction in human superior temporal sulcus. Neuropsychologia. 2007. 45(14): 3234–3241. https://doi.org/10.1016/j.neuropsychologia.2007.06.022
  21. Frith U. A new look at language and communication in autism. International Journal of Language & Communication Disorders. 1989. 24(2): 123–150. https://doi.org/10.3109/13682828909011952
  22. Giese M.A., Rizzolatti G. Neural and Computational Mechanisms of Action Processing: Interaction between Visual and Motor Representations. Neuron. 2015. 88(1): 167–180. https://doi.org/10.1016/j.neuron.2015.09.040
  23. Gosch S., Puhlmann L.M.C., Lauckner M.E., Förster K., Kanske P., Wiesmann C.G., Preckel K. An fMRI study on alexithymia and affective state recognition in the Reading the Mind in the Eyes Test. Social Cognitive and Affective Neuroscience. 2024. 19(1): nsae058. https://doi.org/10.1093/scan/nsae058
  24. Greenberg D.M., Warrier V., Abu-Akel A., Allison C., Gajos K.Z., Reinecke K., Rentfrow P.J., Radecki M.A., Baron-Cohen S. Sex and age differences in “theory of mind” across 57 countries using the English version of the “Reading the Mind in the Eyes” Test. Proceedings of the National Academy of Sciences of the United States of America. 2023. 120(1): e2022385119. https://doi.org/10.1073/PNAS.2022385119/SUPPL_FILE/PNAS.2022385119.SAPP.PDF
  25. Gunther Moor B., Op de Macks Z.A., Güroǧlu B., Rombouts S.A.R.B., Van der Molen M.W., Crone E.A. Neurodevelopmental changes of reading the mind in the eyes. Social Cognitive and Affective Neuroscience. 2012. 7(1): 44–52. https://doi.org/10.1093/SCAN/NSR020
  26. Handley G., Kubota J.T., Li T., Cloutier J. Black “Reading the Mind in the Eyes” task: The development of a task assessing mentalizing from black faces. PLOS ONE. 2019. 14(9): e0221867. https://doi.org/10.1371/JOURNAL.PONE.0221867
  27. Hartshorne J.K., Germine L.T. When Does Cognitive Functioning Peak? The Asynchronous Rise and Fall of Different Cognitive Abilities Across the Life Span. Psychological Science. 2015. 26(4): 433–443. https://doi.org/10.1177/0956797614567339
  28. Hess U., Blairy S. Facial mimicry and emotional contagion to dynamic emotional facial expressions and their influence on decoding accuracy. Int J Psychophysiol. 2001. 40(2): 129–141. https://doi.org/10.1016/s0167-8760(00)00161-6
  29. Hugdahl K., Raichle M.E., Mitra A., Specht K. On the existence of a generalized non-specific task-dependent network. Front. Hum. Neurosci. 2015. 9: https://doi.org/10.3389/fnhum.2015.00430
  30. Johnson B.N., Kivity Y., Rosenstein L.K., LeBreton J.M., Levy K.N. The Association BetweenMentalizing and Psychopathology: AMeta-Analysis of the Reading theMind in the Eyes Task Across Psychiatric Disorders. Clinical Psychology: Science and Practice. 2022. 29(4): 423–439. https://doi.org/10.1037/CPS0000105
  31. Kamashita R., Setsu R., Numata N., Koga Y., Nakazato M., Matsumoto K., Ando H., Masuda Y., Maral S., Shimizu E., Hirano Y. Atypical social cognition processing in bulimia nervosa: an fMRI study of patients thinking of others’ mental states. BioPsychoSocial Medicine. 2024a. 18(1): 1–12. https://doi.org/10.1186/S13030-023-00297-Y/TABLES/4
  32. Kim H.A., Kaduthodil J., Strong R.W., Germine L.T., Cohan S., Wilmer J.B. Multiracial Reading the Mind in the Eyes Test (MRMET): An inclusive version of an influential measure. Behavior Research Methods. 2024. 1–18. https://doi.org/10.3758/S13428-023-02323-X/FIGURES/8
  33. Koo S.J., Kim Y.J., Han J.H., Seo E., Park H.Y., Bang M., Park J.Y., Lee E., An S.K. “Reading the Mind in the Eyes Test”: Translated and Korean Versions. Psychiatry Investigation 2021. 18(4): 295. https://doi.org/10.30773/PI.2020.0289
  34. Leslie K.R., Johnson-Frey S.H., Grafton S.T. Functional imaging of face and hand imitation: towards a motor theory of empathy. Neuroimage. 2004. 21(2): 601–607. https://doi.org/10.1016/j.neuroimage.2003.09.038
  35. Malatesta G., D’Anselmo A., Prete G., Lucafò C., Faieta L., Tommasi L. The Predictive Role of the Posterior Cerebellum in the Processing of Dynamic Emotions. Cerebellum. 2024. 23(2): 545–553. https://doi.org/10.1007/s12311-023-01574-w
  36. Mar R.A. The Neural Bases of Social Cognition and Story Comprehension. Annual Review of Psychology. 2011. 62(1): 103–134. https://doi.org/10.1146/annurev-psych-120709-145406
  37. Masharipov R., Knyazeva I., Korotkov A., Cherednichenko D., Kireev M. Comparison of whole-brain task-modulated functional connectivity methods for fMRI task connectomics. Commun Biol. 2024. 7(1): 1–21. https://doi.org/10.1038/s42003-024-07088-3
  38. Molenberghs P., Cunnington R., Mattingley J.B. Brain regions with mirror properties: a meta-analysis of 125 human fMRI studies. Neurosci Biobehav Rev. 2012. 36(1): 341–349. https://doi.org/10.1016/j.neubiorev.2011.07.004
  39. Molenberghs P., Johnson H., Henry J.D., Mattingley J.B. Understanding the minds of others: A neuroimaging meta-analysis. Neuroscience & Biobehavioral Reviews. 2016. 65: 276–291. https://doi.org/10.1016/j.neubiorev.2016.03.020
  40. Narumoto J., Okada T., Sadato N., Fukui K., Yonekura Y. Attention to emotion modulates fMRI activity in human right superior temporal sulcus. Brain Res Cogn Brain Res. 2001. 12(2): 225–231. https://doi.org/10.1016/s0926-6410(01)00053-2
  41. Nolte T., Bolling D.Z., Hudac C.M., Fonagy P., Mayes L., Pelphrey K.A. Brain mechanisms underlying the impact of attachment-related stress on social cognition. Front Hum Neurosci. 2013. 7: 816. https://doi.org/10.3389/fnhum.2013.00816
  42. Oldfield R.C. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971. 9(1): 97–113. https://doi.org/10.1016/0028-3932(71)90067-4
  43. Overgaauw S., van Duijvenvoorde A.C.K., Gunther Moor B., Crone E.A. A longitudinal analysis of neural regions involved in reading the mind in the eyes. Soc Cogn Affect Neurosci. 2015a. 10(5): 619–627. https://doi.org/10.1093/scan/nsu095
  44. Pavlova M.A., Sokolov A.A. Reading language of the eyes. Neuroscience & Biobehavioral Reviews. 2022. 140: 104755. https://doi.org/10.1016/J.NEUBIOREV.2022.104755
  45. Peñuelas-Calvo I., Sareen A., Sevilla-Llewellyn-Jones J., Fernández-Berrocal P. The “Reading the Mind in the Eyes” Test in Autism-Spectrum Disorders Comparison with Healthy Controls: A Systematic Review and Meta-analysis. Journal of Autism and Developmental Disorders. 2019. 49(3): 1048–1061. https://doi.org/10.1007/S10803-018-3814-4/FIGURES/5
  46. Pierce J.E., Thomasson M., Voruz P., Selosse G., Péron J. Explicit and Implicit Emotion Processing in the Cerebellum: A Meta-analysis and Systematic Review. Cerebellum (London, England). 2022. 22(5): 852. https://doi.org/10.1007/s12311-022-01459-4
  47. Premack D., Woodruff G. Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences. 1978. 1(4): 515–526. https://doi.org/10.1017/S0140525X00076512
  48. Preti A., Siddi S., Marzola E., Abbate Daga G. Affective cognition in eating disorders: a systematic review and meta-analysis of the performance on the “Reading the Mind in the Eyes” Test. Eating and Weight Disorders – Studies on Anorexia, Bulimia and Obesity. 2022 27(7): 2291–2307. https://doi.org/10.1007/S40519-022-01393-8
  49. Quidé Y., Wilhelmi C., Green M.J. Structural brain morphometry associated with theory of mind in bipolar disorder and schizophrenia. PsyCh journal. 2020. 9(2): 234–246. https://doi.org/10.1002/PCHJ.322
  50. Rizzolatti G., Craighero L. THE MIRROR-NEURON SYSTEM. Annual Review of Neuroscience. 2004. 27(Volume 27, 2004): 169–192. https://doi.org/10.1146/annurev.neuro.27.070203.144230
  51. Sabatinelli D., Fortune E.E., Li Q., Siddiqui A., Krafft C., Oliver W.T., Beck S., Jeffries J. Emotional perception: meta-analyses of face and natural scene processing. Neuroimage. 2011. 54(3): 2524–2533. https://doi.org/10.1016/j.neuroimage.2010.10.011
  52. Schmidtmann G., Jennings B., Sandra D., Pollock J., Gold I. The McGill Face Database: Validation and Insights Into the Recognition of Facial Expressions of Complex Mental States. Perception. 2020. 49: 1–20. https://doi.org/10.1177/0301006620901671
  53. Schraa-Tam C.K.L., Rietdijk W.J.R., Verbeke W.J.M.I., Dietvorst R.C., van den Berg W.E., Bagozzi R.P., De Zeeuw C.I. fMRI activities in the emotional cerebellum: a preference for negative stimuli and goal-directed behavior. Cerebellum. 2012. 11(1): 233–245. https://doi.org/10.1007/s12311-011-0301-2
  54. Schurz M., Radua J., Aichhorn M., Richlan F., Perner J. Fractionating theory of mind: A meta-analysis of functional brain imaging studies. Neuroscience and Biobehavioral Reviews. 2014. 42: 9–34. https://doi.org/10.1016/J.NEUBIOREV.2014.01.009
  55. Schurz M., Radua J., Tholen M.G., Maliske L., Margulies D.S., Mars R.B., Sallet J., Kanske P. Toward a hierarchical model of social cognition: A neuroimaging meta-analysis and integrative review of empathy and theory of mind. Psychol Bull. 2021. 147(3): 293–327. https://doi.org/10.1037/bul0000303
  56. Shamay-Tsoory S.G., Shur S., Barcai-Goodman L., Medlovich S., Harari H., Levkovitz Y. Dissociation of cognitive from affective components of theory of mind in schizophrenia. Psychiatry Research. 2007. 149(1–3): 11–23. https://doi.org/10.1016/J.PSYCHRES.2005.10.018
  57. Simon M., Németh N., Gálber M., Lakner E., Csernela E., Tényi T., Czéh B. Childhood Adversity Impairs Theory of Mind Abilities in Adult Patients With Major Depressive Disorder. Frontiers in Psychiatry. 2019. 10(2): 471216. https://doi.org/10.3389/FPSYT.2019.00867/BIBTEX
  58. Sliwinska M.W., Elson R., Pitcher D. Dual-site TMS demonstrates causal functional connectivity between the left and right posterior temporal sulci during facial expression recognition. Brain Stimul. 2020. 13(4): 1008–1013. https://doi.org/10.1016/j.brs.2020.04.011
  59. Stafford O., Gleeson C., Egan C., Tunney C., Rooney B., O’Keeffe F., McDermott G., Baron-Cohen S., Burke T. A 20-Year Systematic Review of the ‘Reading the Mind in the Eyes’ Test across Neurodegenerative Conditions. Brain Sciences. 2023. 13(9): 1268. https://doi.org/10.3390/BRAINSCI13091268/S1
  60. Thye M.D., Bednarz H.M., Herringshaw A.J., Sartin E.B., Kana R.K. The impact of atypical sensory processing on social impairments in autism spectrum disorder. Dev Cogn Neurosci. 2018a. 29: 151–167. https://doi.org/10.1016/j.dcn.2017.04.010
  61. Thye M.D., Murdaugh D.L., Kana R.K. Brain Mechanisms Underlying Reading the Mind from Eyes, Voice, and Actions. Neuroscience. 2018b. 374: 172–186. https://doi.org/10.1016/j.neuroscience.2018.01.045
  62. Tollenaar M.S., Overgaauw S. Empathy and mentalizing abilities in relation to psychosocial stress in healthy adult men and women. Heliyon. 2020. 6(8): e04488. https://doi.org/10.1016/j.heliyon.2020.e04488
  63. Van Overwalle F. Social and emotional learning in the cerebellum. Nat. Rev. Neurosci. 2024. 1–16. https://doi.org/10.1038/s41583-024-00871-5
  64. Van Overwalle F. Social cognition and the brain: A meta-analysis. Human Brain Mapping 2009. 30(3): 829–858. https://doi.org/10.1002/HBM.20547
  65. Van Overwalle F., Haihambo N., Ma Q., Li M., Martínez-Regueiro R., Argoub I., Firouzi M., Deroost N., Baeken C., Baetens K. Cerebellar non-invasive stimulation of social and emotional mentalizing: A meta-analysis. Imaging Neuroscience. 2024. https://doi.org/10.1162/imag_a_00334
  66. Van Overwalle F., Manto M., Cattaneo Z., Clausi S., Ferrari C., Gabrieli J.D.E., Guell X., Heleven E., Lupo M., Ma Q., Michelutti M., Olivito G., Pu M., Rice L.C., Schmahmann J.D., Siciliano L., Sokolov A.A., Stoodley C.J., van Dun K., Vandervert L., Leggio M. Consensus Paper: Cerebellum and Social Cognition. Cerebellum. 2020. 19(6): 833–868. https://doi.org/10.1007/s12311-020-01155-1
  67. Vucurovic K., Caillies S., Kaladjian A. Neural correlates of theory of mind and empathy in schizophrenia: An activation likelihood estimation meta-analysis. Journal of Psychiatric Research. 2020. 120: 163–174. https://doi.org/10.1016/J.JPSYCHIRES.2019.10.018
  68. Woolley A.W., Aggarwal I., Malone T.W. Collective Intelligence and Group Performance. Current Directions in Psychological Science. 2015. 24(6): 420–424. https://doi.org/10.1177/0963721415599543
  69. Yin S., Fu C., Chen A. The structural and functional correlates underlying individual heterogeneity of reading the mind in the eyes. Biol Psychol. 2018. 138: 179–184. https://doi.org/10.1016/j.biopsycho.2018.09.009

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Schematic illustration of the Reading the Mind in the Eyes Test (RMET).

Download (70KB)
Indexing metadata
3. Fig. 2. Statistical parametric maps for contrasts «Affective mental state recognition > Age recognition” and “Age recognition > Affective mental state recognition» (cluster-level pFWE < 0.05). For each cluster, the effect size is presented (in arbitrary units): blue for the affective mental state recognition, pink – for the age recognition.

Download (446KB)
Indexing metadata
4. Fig. 3. Comparisons of results for the current study («Affective mental state recognition > Age recognition» contrast») and a recent meta-analysis of Schurz et al. (Schurz et al., 2021).

Download (230KB)
Indexing metadata

Copyright (c) 2025 Russian Academy of Sciences