Do you ever stop to think about what is happening when you imagine something? Or when you picture something perfectly in your mind’s eye? The process of creating images or thoughts in our head the way we have never actually experienced physically is both fascinating and perplexing.
Andrey Vyshedskiy, neuroscientist and adjunct professor at MET, explores these concepts in the second edition of On the Origin of the Human Mind. In his book, Professor Vyshedskiy examines the origins of the human mind from a scientific and neurological perspective and offers various tests and theories about how the mind works. In the following Q&A with Professor Voices, Professor Vyshedskiy offers some insight about the concept of mental synthesis in particular.
Professor Voices: What is mental synthesis?
Andrey Vyshedskiy: Consider the puzzle below. Can you find the patch that completes the image?
Your success in solving this and similar puzzles depends on your ability to mentally synthesize two objects that you have never seen together before. The neurological mechanism that allows you to envision the solution to the puzzle is the same one that allows you to actively imagine any novel object or scene, and is thus at the core of human imagination and creativity. To distinguish this active type of imagination from other aspects of mental imagery such as simple memory recall, dreaming, hallucinations, and spontaneous insight, we call it mental synthesis.
PV: What are some things that we think about every day that show mental synthesis at work?
Andrey Vyshedskiy: mental synthesis permeates through just about every aspect of human behavior and is an integral part of many of our everyday tasks. When we speak we use mental synthesis to communicate a novel image (“My house is the second one on the left, just across the road from the red gate”) and we rely on the listener to use mental synthesis to make sense of our words and follow our instructions. When we tell stories, we are often describing things that the listener has never seen before (“That creature has three heads, two tails and can run faster than a cheetah”) and we rely on the listener to imagine the story in their mind’s eye. When we teach, we do so actively, using language (“Imagine you have 5 cookies. You eat 2 of them. How many do you have left?”) and we rely on the student’s ability to use mental synthesis to mentally simulate the instructions, construct a new mental image, and arrive at an answer. As kids, we exhibit our mental synthesis through pretend play, by creating and playing within worlds of our own imagination. As we grow older, our games become more complex. We continue exercising our mental synthesis through games such as chess, Sudoku and crosswords. Representational art, scientific innovation, as well as design and construction are just some of the creative manifestations of mental synthesis.
PV: What is the neurobiological basis of the mental synthesis theory?
AV: At the basis of the mental synthesis theory lies a simple, yet fundamental question: what happens neurologically when two objects, never before seen together (say, an apple on top of a dolphin), are imagined together for the first time? The scientific consensus is that a familiar object, such as an apple or a dolphin, is represented in the brain by thousands of neurons dispersed throughout the posterior cortex (occipital, temporal, and parietal lobes). When one sees or recalls such an object, the neurons of that object’s neuronal ensemble tend to activate into synchronous resonant activity. The neuronal ensemble binding mechanism, based on the Hebbian principle “neurons that fire together, wire together,” came to be known as the binding-by-synchrony hypothesis. However, while the Hebbian principle explains how we perceive a familiar object, it does not explain the infinite number of novel objects that humans can voluntarily imagine. The neuronal ensembles encoding those objects cannot jump into spontaneous synchronized activity on their own since the parts forming those novel images have never previously been seen together. The mental synthesis theory posits that to account for unlimited human imagination, the binding-by-synchrony hypothesis would need to be extended to include the phenomenon of mental synthesis whereby the prefrontal cortex ACTIVELY and INTENTIONALLY synchronizes independent neuronal ensembles into one morphed image. Thus, the apple neuronal ensemble is synchronized with the dolphin neuronal ensemble, and the two disparate objects are perceived together for the first time. The synchronization mechanism of mental synthesis is likely responsible for many imaginative and creative traits that philosophers and scientists have recognized as being uniquely human, despite not having a precise neurological understanding of the process.
PV: What role does language play in mental synthesis?
AV: Research on early brain development shows that the development of mental synthesis depends on the timely acquisition of syntactic speech during childhood. Children talk with themselves and others and in the process create an infinite number of novel mental images in their brain. The active use of syntactic language during childhood likely naturally leads to the development of synchronous neural connections between the prefrontal cortex and the posterior cortex. It is the synchronous connections that seem to be essential for mental synthesis. Children not involved in syntactic communication during the sensitive period of language development often find it challenging to intentionally create and test new scenarios in their mind. They would, for example, have great difficulty solving the above elephant puzzle, or doing any tasks that require an active imagination. In a sense, these children develop a “mental synthesis disability” that stays with them for the remainder of their lives. Extreme cases are exemplified by feral children, who grow up without hearing any language, and deaf linguistic isolates, who grow up communicating to their families using homesign, a system of gestures which allows them to communicate simple commands, but lacks much in the way of syntax. This is typical of families with deaf children, which are isolated from a sign language community; instead of learning a formal sign language, they normally spontaneously develop a homesign system. These individuals often fail in the most basic of mental synthesis tests such as being able to follow the directions of “placing the bowl behind/in front of/in/on/under/beside the cup.” To correctly place a bowl behind or in front of a cup, one first needs to mentally synthesize the novel image of a bowl behind or in front of a cup. Someone who cannot simulate the process mentally would have no mental image of a bowl behind or in front of a cup and would therefore just use trial and error and probably place the cup and bowl into an incorrect arrangement. This test is easy for an average four-year-old child, but it is too difficult for adults who were never exposed to syntactic language during their childhood. Research shows that after the end of the critical period, these linguistically deprived individuals are able to learn the meanings of new words, but they are unable to easily generate novel mental scenarios, mentally simulate a plan, or understand complex syntax.
PV: Are some of us better than others at mental synthesis? If so, what contributes to that variation?
AV: As I mentioned above, early exposure to syntactic language is essential for the acquisition of mental synthesis. In addition to this cultural input, acquisition of mental synthesis is influenced by genetics. For example, the duration of the prefrontal cortex plasticity is dramatically prolonged in modern humans compared to other primates. The level of maturation of the prefrontal cortex that chimpanzees attain in a few months after birth is extended to 4-10 years in humans. This difference in the maturation schedule of the prefrontal cortex may explain why chimpanzees exposed to syntactic language from birth learn the meaning of hundreds of words but do not acquire syntactic language that could indicate the presence of mental synthesis. It remains to be seen if natural variation in the maturation schedule of the prefrontal cortex in modern humans affects their propensity for mental synthesis.
PV: Could you provide a general overview of this book’s topic and your theory?
AV: While the mental synthesis theory has predictions reaching as far as neuroscience of perception and childhood education, its main goal was to understand the evolution of the human mind. Acquisition of language is commonly cited as the reason that humans dramatically changed their behavior around 100,000 years ago. However, to this day, there is vigorous scientific discussion around the following two questions: what part of language was acquired 100,000 years ago and why was language acquired so abruptly? Charles Darwin envisioned the origin of language simply as the acquisition of a mechanical ability to produce sounds, which provides the basis for a communication system. A modified version of that hypothesis claims that a culturally acquired communication system enabled language acquisition. Opponents of the “nurture” hypothesis argue that human syntactic language cannot be taught to animals and therefore humans must be unique in their genetic predisposition to language. Spearheading the “nature” hypothesis is Noam Chomsky who argues that a genetic mutation, which took place 100,000 years ago, enabled the innate faculty of language. Both parties agree that hominin behavior changed dramatically some time around 100,000 years ago, but the nurture party argues that a communication system was culturally acquired at that time, while the nature party argues that a genetic mutation predisposed humans to language acquisition. Neither hypothesis, however, adequately explains the dramatic and sudden change in human behavior: the appearance of representational drawings and sculpture, unprecedented technological progress, and lightning-fast expansion of settlements to all habitable regions of the planet. It is relatively easy to imagine how a single mutation could have increased the brain volume by 10% or the number of synapses by 10% or the number of glial cells by 10% but it is unclear how that relatively small change in neurology could have resulted in such an abrupt change of behavior. If it was not a genetic mutation but solely the acquisition of a communication system that was acquired 100,000 years ago, then it is unclear why the communication system that had been developing for close to two million years (as evidenced by the developing speech apparatus) generated such a dramatic change in behavior 100,000 years ago but not earlier. My analysis shows that the nurture and nature theories are not mutually exclusive and that, in fact, both theories are correct. As of 100,000 years ago, hominins had already evolved both a greater control of perception by the prefrontal cortex and a nearly modern speech-production apparatus. However the connections between the prefrontal cortex and the posterior cortex remained asynchronous; the prefrontal cortex was unable to synchronize independent neuronal ensembles, the communication system remained finite and non-syntactic: one word was only able to communicate one image. At that time, a single mutation delayed the ontogenetic development of the prefrontal cortex. This mutation must have occurred concurrently in two individuals, most likely twins or siblings, who were then able to provide each other with the necessary training to develop synchronous connections between the prefrontal cortex and the posterior cortex. This training would have occurred through conversations between the two individuals, and would have hinged on the invention of a few spatial prepositions. Lets refer to these individuals as Romulus and Remus, and let’s take a closer look at the difference between their newly invented language and the language of their ancestors. The parents of Romulus and Remus had a modern speech apparatus, but without mental synthesis, they could not use a flexible syntax, verb tenses or spatial prepositions. Their communication system was finite and non-syntactic. In that communication system, 1,000 nouns were only capable of communicating 1,000 images to a listener. As we have seen in linguistically deprived children, such finite communication systems (such as homesign) are unable to fine-tune synchronous connections between the prefrontal cortex and the posterior cortex. However, adding just two spatial prepositions instantaneously converts a finite communication system to an infinite syntactic communication system. A communication system with 1,000 nouns and just two spatial prepositions drastically increases the number of distinct images that can be communicated to a listener. With a vocabulary of 1,000 nouns and two spatial prepositions, and sticking only to phrases with three nouns and two spatial prepositions (such as “a bowl on a cup behind a plate”), the number of distinct images increases from one thousand to four billion (1,000 x 2 x 1,000 x 2 x 1,000 = 4,000,000,000). With invention of just a few spatial prepositions, Romulus and Remus’ communication system was instantaneously converted to an infinite syntactic communication system and therefore was able to fine-tune synchronous connections. In this way, Romulus and Remus ontogenetically acquired mental synthesis – an ability to imagine any number of novel objects and to mentally simulate any plan. This gave Romulus and Remus a huge advantage over their peers. Romulus and Remus were the first behaviorally modern humans. They had both components of the full human language: an infinite syntactic communication system and mental synthesis. They were able to teach the syntactic communication system to their children and thus transmit language over generations.
PV: What is the current status of research on mental synthesis? Do you believe enough attention is being paid to the phenomenon in the neuroscience world?
AV: Numerous scientists and philosophers have noticed that humans are significantly more imaginative and creative than animals. Lev Vygotsky, a pioneering psychologist of early 20th century claims, “Imagination is a new formation that is not present in the consciousness of the very young child, is totally absent in animals, and represents a specifically human form of conscious activity”. Ian Tattersall, the eminent paleoanthropologist and curator at the American Museum of Natural History writes, “… if there is one single thing that distinguishes humans from other life-forms, living or extinct, it is the capacity for symbolic thought: the ability to generate complex mental symbols and to manipulate them into new combinations. This is the very foundation of imagination and creativity: of the unique ability of humans to create a world in the mind…”. In neuroscience circles, however, studying or even discussing imagination is much less popular. Owing to the subjective nature of the process, imagination is difficult to study in a precise way using neurological techniques. Researchers rarely enter a live human brain with electrodes except in special cases such as during epilepsy treatment. The usual scientific backup — animal subjects — are ill-suited for studying the mechanism of a uniquely human phenomenon. With the invention of the functional MRI technique in the early 1990s, scientists gained the ability to peek inside the human brain noninvasively, but fMRI remains a crude method for studying imagination. Its low spatial and temporal resolution are not very useful for distinguishing mental synthesis from simple memory recall. Since both phenomena are mediated by overlapping cortical areas, fMRI is largely inept for revealing the neurological differences between the two phenomena. Furthermore the time resolution of fMRI is prohibitively low to study the effect of synchronization. As a result, mental synthesis and simple memory recall remained bound together under the title of mental imagery; mental synthesis has never made a separate entry into the realm of neuroscience. I am trying to change that.
PV: Is there any criticism of the mental synthesis theory? If so, how do you support your claim?
AV: The book has received many positive reviews from prominent scholars. As was the case with the first edition of the book, the most counterintuitive part of the mental synthesis theory remains its prediction that non-human animals lack the neurological machinery for mental synthesis. For some people it is hard to accept that their dog, who can tell the ball from the bowl, cannot voluntarily visualize the ball behind, in front of, in, under, or beside the bowl. The mental synthesis theory tightly binds an infinite syntactic communication system to mental synthesis: until you have one, you don’t have the other. This prediction is supported by two lines of circumstantial evidence: observations of mental synthesis disability in linguistically deprived children provide an ontogenetic confirmation and the archeological findings provide a phylogenetic confirmation. All four external manifestations of mental synthesis identifiable in the archeological record: religious beliefs inferred through adorned burials, creativity and innovation, design and construction, and representational art appear in the archeological record only about 40,000 years ago and are associated with modern humans. The reason this prediction remains counterintuitive to some readers is that humans are wired for anthropomorphic beliefs. We think that other animals, especially other mammals, think just as we do. We interpret the intentions of other animals by imagining what we would do and think in their place. In fact, Heider & Simmel demonstrated, over 70 years ago, that this interpretation mechanism is extended to most animate objects, even geometric shapes moving in ways that an animal would move. Yes, some animals can recall images from memory, maybe even sequences of images like movie clips, but they likely cannot voluntarily change the sequence of frames, reposition the actors or reset the scenery the way humans can.
PV: Do you believe mental synthesis in humans has reached a plateau? Or do you see it continuing to evolve over time?
AV: Improving early education has a great potential to advance mental synthesis in most children. As I mentioned above, the development of the mechanism responsible for mental synthesis seems to have a critical period after which further development is impossible. This is not unusual for a neurological system. There is no doubt that neural plasticity in many systems is enhanced during specific windows of opportunity early on in childhood and diminishes greatly at the end of the critical period. Critical periods have been found to exist in virtually all species, from humans to Drosophila. For example, in mammals, artificial closure of one eye for the duration of the critical period causes a permanent loss of vision through that eye. Loss of vision occurs despite there being no damage to the sensory receptors in the eye, the thalamus, or the cerebral cortex. Remarkably, the simple act of covering an eye can profoundly alter the physical structure of the brain. Of course the biggest difference between normal development of vision through an eye and the acquisition of mental synthesis is that light reflected from surrounding objects reaches the retina naturally whenever it is light. However, the development of mental synthesis requires a community of humans willing to engage a child with the use of a syntactic language. Only exposure to a syntactic language seems to provide the adequate input for the development of mental synthesis. Furthermore, the exposure to a syntactic language has to occur during the period of neural plasticity, which expires shortly before puberty. As puberty can occur at an early age, it is important to insure that children are engaged in conversations as early as possible.
PV: Do you have any new projects or research relating to mental synthesis coming in the future?
AV: Recently, Simon (the name has been changed for privacy reasons), a friend’s son who I have known from the time he was a child, was institutionalized. He was diagnosed with autism and had a significant language delay. Language delay is a common affliction that affects 70% of children with autism, a majority of children with Down syndrome and thousands of other children. One of the main challenges of working with these children is finding a way to provide them with the necessary “brain training” without the use of language. These children are not able to benefit from the training provided naturally by a syntactic communication system at the time when their brains are most receptive to the development of mental synthesis. As a result, many children with language delay end up with a significant impairment in the ability to intentionally imagine novel scenarios and to mentally solve even the simplest of problems. For these children it is often very difficult to imagine the consequences of their actions and to understand the pain they could inflict on others. Simon gradually became more and more aggressive and his parents had no choice but to institutionalize him.
I was wondering if the tragedy of Simon’s institutionalization could have been prevented by timely exposure to some brain training exercises that could have helped Simon in the acquisition of mental synthesis. How many kids could avoid being institutionalized and could remain active and productive members of society if only we could find a way to help them train their mental synthesis before the end of the critical period? It came to my attention that a methodology based exclusively on visual puzzles similar to the one at the start of the interview, may be well suited for such a task. This thinking resulted in the development of ImagiRation software. In the absence of speech, ImagiRation software provides exercises designed to facilitate the development of mental synthesis. The software aims to provide a non-verbal approach to train the brain to imagine never-before-seen images and scenarios. The program is designed as an intelligent tutoring system, dynamically responding and adjusting to the needs of each individual child. Since the software is aimed at kids with a language disorder, it is designed to require minimal instruction and explanation, working more as a facilitator; kids work independently to figure out the solution without verbal instruction. The software’s main job is to adaptively adjust the difficulty of problems to be just one small step away from the child’s level (not too easy and not too difficult). ImagiRation approach creates a non-verbal way for kids to acquire mental synthesis independently. Our belief is that, as long as the staircase is adjusted correctly, any child can climb up. My hope is that the timely application of the ImagiRation visual puzzles can save millions of children with language delay from institutionalization and help them grow into fully integrated members of society.
Free electronic version of “On The Origin Of The Human Mind,” 2nd edition is available at mobilereference.com/mind