The erratic embodied nature of translation: a neuroscientific perspective

How does translation work? What happens in our brain when we switch between languages? Martina Ardizzi, a neuropsychologist and researcher at the University of Parma, sheds light on the mechanisms at work in our brain when we deal with different languages.

he history of neuroscience could not be written without mentioning the study of human language in the opening chapters. The first steps taken by philosophers and physicians to discover the functioning of our mysterious brain were made possible by studying patients with language disorders. 

The very first medical treatise containing a reference to the brain in the history of humankind is a papyrus dating back to the 17th century BC. According to James Breasted, who translated and published the document in 1930, the word “brain” occurs only eight times in ancient Egypt, with six of these occurrences coming in this papyrus. It is therefore more than a mere coincidence that this archaeological find contains a description of the earliest known case of an aphasic patient. From the 17th century BC to much of the 20th century AD, a great deal of knowledge about the functioning of the human brain was almost entirely the result of accident (in the most literal sense of the word). The history of neuroscience unfolds in descriptions of patients whose linguistic production after severe brain injuries is restricted to the syllable “tan” or others whose logorrhoea is an unintelligible succession of paraphasia and neologisms. Observing the behaviour of patients with specific brain lesions allowed researchers to functionally correlate certain mental faculties with specific areas of the brain, and this was particularly true for language. The same fate was in store for the study of the neurobiological mechanisms involved in translating. In 1984, Ellen Perecman described the interesting case of H.B., an 80-year-old male born in Cameroon to German parents, who developed polyglot aphasia following extensive bilateral temporal haematomas. H.B. learned German as a native language, French as a second language, and then English when he settled in the United States at the age of 18. After the injury, H.B.‘s language was strongly marked by language mixing. The patient shifted from one language to another during a single conversation. He frequently spontaneously translated his own utterances even if he could not translate upon request. In the last few years of his life, H.B.’s speech continually strayed between the different linguistic territories he had travelled to in his life (H.B.: was wollte ich sagen/nichts besondereslwhat could I say/nothing special).

---

---

In recent decades, powerful new techniques for observing brain function in healthy living individuals have been devised. To a large extent, the neuroscience of language has become a rapidly developing field because of new technologies that allow us to monitor the “speaking brain” as it actually works, to document neural pathways, and even to track the activity of specific neurons involved in this extraordinary human faculty. However, the neurological mechanisms involved in translating remain one of the chief “known unknowns”. We know very little about how a translator’s brain activity relates to specific aspects of translation. Much of this terra incognita can be illuminated by moving the field beyond the parochial presuppositions of disciplines bringing together translation scholars, cognitive scientists, computer scientists, literacy and language experts, and also neuroscientists. My aim in this piece is to claim a place at the discussion table by providing some insights into what neuroscience can offer for reflecting on the future of translation in a world in which understanding one another and making oneself understood require us to constantly cross increasingly dynamic and blurred cultural and linguistic boundaries. 

The paucity of neuroscientific studies on translation is quite surprising if we consider that the brain is a prodigious translator. Any sensory information from outside or inside the body is translated into an electrical signal and then back into a specific phenomenological experience. Myriad photons reach our retina, where they are transduced into electrical signals, but what we see are lights, colours and shapes. The letters our brain uses are simple electrical signals, but what it is talking about is our first-hand experience. This is possible because there is a “brain-body system” responsible for the integration of signals and the construction of a subjective experience. The neuroscientific field has moved through several models or paradigms explaining the nature of the mind. Computational models, according to which cognition and consciousness are a form of computation, have nowadays given way to an embodied model. The human mind is no longer considered as an abstract and isolated entity. Instead, the mind is integrated into the brain-body sensorimotor system. Cognitive processes, including language and translation, are mediated by the brain-body system. 

Neuroscience has unveiled brain patterns behind several cognitive capacities grounded in action and perception within the body. In this context, language and translation are no exception. When participants lie quietly in an operational MRI scanner and simply read action words such as “kick” or “pick”, they spontaneously activate portions of their brain’s motor cortices, specifically those controlling leg versus hand movements. Similarly, reading odour words such as “jasmine” or “cinnamon” elicits activity in the olfactory regions of the brain. In the case of native language words with a high number of visual associations, such as the word ‘‘apple’’, brain responses in visual areas occur even before the completion of lexical access to the word. We first grasp the concept “apple” through the apple-related experiences stored in our brain, and then we get access to the “symbolic” apple. 

If words were symbols, totally abstract entities with no connection to the body, reading them should not activate sensorimotor brain regions related to the direct experience of these actions, smells and objects. Therefore, in the brain’s language, a word is represented as a sensorimotor network that mirrors all experiences connected to the concept. Strikingly, abstract concepts have also proved to be embodied and experience-related, even if in this case they are connected to highly complex experiential clusters that are not focally contained in a single brain region and so are more difficult to localise. It is important to note that this sensorimotor access to linguistic concepts supports language comprehension. Following the embodied approach to language, recent studies demonstrate that translation is also strongly grounded in our brain-body system. Learning new foreign words while performing gestures is more efficient than the common practice of learning with pictures. Furthermore, the more motor regions are activated during the translation of foreign words, the more accurate the translator’s performance will be. By disturbing the activity of the primary motor area during the translation of foreign words learned in association with hand gestures, Mathias and colleagues finally demonstrate the causal contribution of motor regions to the translation of foreign vocabulary. 

The involvement of sensorimotor processes is not reduced to the mere solipsistic translation of words into a sort of empirical vacuum. Language in general and translation specifically serve a social communication purpose and are fundamentally social behaviours. Children, and to some extent also adults, depend on social interactions to learn both native and second languages. Computational models that incorporate social-interactive cues perform significantly better than models with no such cues included. Human beings and other animal species are indeed endowed with specific neurobiological mechanisms supporting our social skills, of which mirror mechanisms are certainly the most ancestral and embodied. Briefly, mirror mechanisms are widespread throughout the brain and correspond to the ability of specific sensorimotor or visceromotor neurons or brain regions to be ubiquitously activated during both first-hand experiences and merely observed experiences. The mirror mechanisms use the embodied nature of our experience to rapidly understand what is happening to others. The ability to comprehend others through a direct mapping mechanism in the brain strongly facilitates social life, providing a neural basis for some of the interpersonal relations on which more complex social behaviours like language and translation are built. Essentially, you need a body in order to acquire the sensorimotor and visceromotor experiences that you can then use to interpret actions, emotions and sensations performed or felt by others. 

To sum up, the Cartesian era is over: neither Descartes, nor Kant, nor Fodor was right. Cognition is neither amodal nor symbolic, and the same goes for language and translation. Empirical evidence shows that the brain does not work like a “computer” processing symbols; we need our body and the experiences acquired through it. If you look at intelligence from an evolutionary perspective, you will find that brains have always developed in the context of a body that interacts with the environment. There is no algorithmic ether in which brains arise.

This paradigm is already inspiring and challenging scholars in the realm of artificial intelligence. Several efforts are being made to integrate this evidence into the design of embodied artificial intelligence and information processing in which physical realisation and the physical environment play an unavoidable and essential role. To apply this endeavour in the field of language and translation, what neuroscience suggests is that there is something more to linguistic translation than what has been considered so far, and this “more” pertains to the physical interaction we have with the environment that endows linguistic symbols in the brain with meaning. No approach, paradigm or theory claims to be completely exhaustive. The point is to be astute enough to draw the best questions from the predominant approach of the time. In this sense, my open question at this discussion table is: can we consider a future of translation technology capable of integrating gestures and sensory experiences as a body of data for machine translation?

Bibliography

Perecman, E., Spontaneous translation and language mixing in a polyglot aphasic, Brain and Language 1984

Hauk, O., Johnsrude, I., & Pulvermüller, F., Somatotopic representation of action words in human motor and premotor cortex, Neuron 2004

González, J., Barros-Loscertales, A., Pulvermüller, F., Meseguer, V., Sanjuán, A., Belloch, V., & Ávila, C., Reading cinnamon activates olfactory brain regions, Neuroimage 2006

Lewis, G., & Poeppel, D., The role of visual representations during the lexical access of spoken words, Brain and Language 2014

Pulvermüller, F., Words in the brain’s language, Behavioral and Brain Sciences 1999.

Buccino, G., Colagè, I., Silipo, F., & D’Ambrosio, P., The concreteness of abstract language: an ancient issue and a new perspective, Brain Structure and Function 2019.

Pulvermüller, F., Hauk, O., Nikulin, V. V., & Ilmoniemi, R. J., Functional links between motor and language systems, European Journal of Neuroscience 2005

Mayer, K. M., Yildiz, I. B., Macedonia, M., & von Kriegstein, K., Visual and motor cortices differentially support the translation of foreign language words, Current Biology 2015

Mathias, B., Waibel, A., Hartwigsen, G., Sureth, L., Macedonia, M., Mayer, K. M., & von Kriegstein, K., Motor cortex causally contributes to vocabulary translation following sensorimotor-enriched training, Journal of Neuroscience 2021.

Li, P., & Jeong, H., The social brain of language: grounding second language learning in social interaction, npj Science of Learning 2020

Pfeifer, R., & Bongard, J., How the body shapes the way we think: a new view of intelligence, MIT Press 2006

MacLennan, B. J., Bodies—both informed and transformed embodied computation and information processing. In Information and Computation, Essays on Scientific and Philosophical Understanding of Foundations of Information and Computation 2011

---

Photo credits: Anton Maksimov, Unsplash / Meo, Unsplash / Karlis Reimanis, Unsplash

---