Home » Science » My Synesthesia Transforms Speech into Text I ‘See’ in My Head

My Synesthesia Transforms Speech into Text I ‘See’ in My Head

I spend my days surrounded by thousands of written words, and sometimes I feel as though there’s no escape. That may not seem particularly unusual. Plenty of people have similar feelings. But no, I’m not just talking about my job as a copy editor here at Scientific American, where I edit and fact-check an endless stream of science writing. This constant flow of text is all in my head. My brain automatically translates spoken words into written ones in my mind’s eye. I “see” subtitles that I can’t turn off whenever I talk or hear someone else talking. This same speech-to-text conversion even happens for the inner dialogue of my thoughts.

This mental closed-captioning has accompanied me since late toddlerhood, almost as far back as my earliest childhood memories. And for a long time, I thought that everyone could “read” spoken words in their head the way I do.

What I experience goes by the name of ticker-tape synesthesia. It is not a medical condition—it’s just a distinctive way of perceiving the surrounding world that relatively few people share.

On supporting science journalism

If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

Not much is known about the neurophysiology or psychology of this phenomenon, sometimes called “ticker taping,” even though a reference to it first appeared in the scientific literature in the late 19th century.

Ticker taping is considered a form of synesthesia, an experience in which the brain reroutes one kind of incoming sensory information so that it is processed as another. For example, sounds might be perceived as touch, allowing the affected person to “feel” them as tactile sensations.

As synesthesia goes, ticker taping is relatively uncommon. “There are varieties of synesthesia which really have just been completely under the radar…, and ticker tape is really one of those,” says Mark Price, a cognitive psychologist at the University of Bergen in Norway. The name “ticker-tape synesthesia” itself evokes the concept’s late 19th-century origins. At that time stock prices transmitted by telegraph were printed on long paper strips, which would be torn into tiny bits and thrown from building windows during parades.

Ticker-tape synesthesia is so obscure that some synesthesia researchers, including Price, only became aware of it after coming across anecdotal reports. He likens synesthesia research in general to “exploring a new universe: you just sort of stumble across these planets you didn’t even know existed,” Price says.

Only recently have a few scientists finally begun to study ticker taping in earnest. Their interest has been generated by a desire to learn about the neural connections that make up the brain’s reading networks. Such efforts might help us better understand dyslexia, a neurodevelopmental condition that makes reading and writing difficult.

These studies are expanding the ranks of ticker tapers. Many people with ticker-tape synesthesia only realize that they have it after they learn about the phenomenon from researchers who are recruiting participants for studies. This growing awareness just reiterates the vast range of human experience. “You never know if your perception is a normal perception or if it’s a particular perception that differs from other ones,” says Fabien Hauw, a neurologist and cognitive neuroscientist at the Paris Brain Institute, who has studied ticker taping.

I recognized my own synesthesia for the first time when I was in my mid-20s. During a conversation, I saw one word in my head spelled in Internet slang—I think it was “gr8” instead of “great.” When I mentioned it out loud, I learned that other people didn’t perceive words this way. Like many people with synesthesia, I had assumed that everyone else shared my experience, and I hadn’t realized that there was anything unusual about it—it was just part of my everyday perception of the world around me. This “is often the case with synesthesia, I believe, because it has no real consequence,” says Laurent Cohen, a neurologist and cognitive neuroscientist at the Paris Brain Institute and Hauw’s former Ph.D. adviser. “It’s not an impairment; it’s not a disease.”

Hauw, Cohen and their colleagues have published several recent studies on the experiences of being a ticker taper. In one, they researched the potential benefits and drawbacks of ticker taping in 22 individuals. The researchers found that ticker tapers were faster and generally more accurate in three tasks involving spoken words than control participants who lacked this mental word-streaming ability. The tasks involved counting the number of letters in words, spelling them backward and deciding whether they contained letters with an “ascending” stroke (such as b or d) or a “descending” one (such as p or q). In two other experiments, the participants had to ignore background speech while they respectively decided whether visually presented terms were actual words or pseudowords and pressed a button based on which of two letters they had seen. Surprisingly, the audio of spoken words did not hinder the majority of the ticker tapers from performing these tasks. Although ticker tapers have self-reported difficulty reading when surrounded by people who are talking, they might also become used to the words they constantly perceive and learn to tune those words out to some extent. “They are probably highly trained at focusing their attention,” Cohen says.

In another study, the same researchers looked at how a separate group of 26 ticker tapers perceived words and found a great deal of variation. Most were “associators”; they visualized the words internally or perceived the words to be located behind their eyes, which is roughly how I would describe my own experience. I don’t literally see words in front of me; instead I automatically visualize them in my mind, and if I hear two conversations at once, I see snippets of both in different “areas” of the visual field of my mind’s eye. I see maybe three or four words at a time, and I “read” them in my head. (In contrast, some other ticker tapers have reported that they see words scroll by.) In Cohen and Hauw’s study of 26 ticker tapers, a smaller subset of the participants saw words projected onto the external visual scene, appearing, for instance, near a speaker’s mouth—almost like a speech bubble in a comic. For some, the word stream appeared at the bottom of their visual field like film subtitles. Differences also were noted in the words’ visual attributes and movement in space and the number of words that appeared at one time. “There was a lot of variability depending on the stimuli and the circumstances,” Hauw says.

Price and his colleagues have also studied the variety of ticker-tape experiences. They surveyed 425 people in Norway—some with ticker-tape synesthesia and some without it—and estimated that only about 0.6 to 3.2 percent of the participants had obligatory ticker taping, meaning that they automatically saw all the words that they heard, spoke or thought. Other people reported that they involuntarily perceived the stream of words only occasionally, and some even were able to call up the mental text voluntarily. Further, some participants only saw the closed-captioning for their own thoughts, not for spoken words. “Weaker tickertapers form part of a graded continuity of experience,” the authors wrote. “This extends from obligatory tickertaping … to the kind of vague visualization of short single words that probably most of us can conjure in our mind with some effort.” In that study, ticker tapers did not self-report that they were skilled at backward spelling or letter counting, but “it could be that people who had ticker taping didn’t know that they have those skills,” Price says.

I had never really thought about ticker taping’s potential advantages and downsides before I read these studies, which raised a whole series of questions. Did my synesthesia help me win a spelling bee in school as a kid? Is it a reason why I always scored so high on word memorization quizzes? Maybe ticker taping helped mentally reinforce my studies, especially because I subtitle my own thoughts. Yet I still get hung up on spelling certain words, such as “committee” or “embarrassed” or “vacuum.” And when I see them in my head, they always look fuzzy. As far as the negatives of ticker taping are concerned, I find it hard to tune out other people’s conversations, especially in an open-office setting, because the ticker tape of their dialogue pops up in my mind.

But it doesn’t distract me to the point where I find myself reaching for my headphones all the time either. Like the participants in Hauw and Cohen’s study of 22 ticker tapers, I can concentrate on a task even with my subtitles popping in and out in the background in the same way that a person watching a movie can sometimes ignore subtitles in a language they speak. The subtitles are such an intrinsic part of how I perceive the world that it’s hard for me to imagine life any other way.

Because ticker taping involves spoken words inducing images of written ones, there is some debate over whether it is really a form of synesthesia at all. Synesthesia typically involves wiring one sensory input to a very different one, such as a connection between sound and touch. Ticker taping, by contrast, appears to exist wholly within the realm of language processing. Spoken and visualized words go together in a way that smells and sounds or tastes and colors do not. And for most of human history, most people were illiterate, so ticker taping is a relatively new phenomenon. Jamie Ward, a professor of cognitive neuroscience at the University of Sussex in England, says that same objection could be raised for other types of synesthesia that are more well studied, though. Imagine, he says, that a person with grapheme-color synesthesia affecting their perception of letters of the alphabet—that is, someone who associates particular letters with different colors—has never seen such a letter in their life. Do they still have synesthesia? “I would say that actually [their] brain is wired differently” from unaffected people, he says. “People with synesthesia think and act in a way that is coherent unto themselves.”

Ward has studied synesthesia clustering—the relationships among different types of synesthesia based on how likely they are to show up in the same individual. In a study he co-authored, ticker taping was one of several more prevalent forms of synesthesia that didn’t cluster with any other type. Part of the reason for this, however, may be that the study split up synesthesia into 164 subtypes that were used for grouping these relationships. Seeing colored letters clustered with seeing colored numbers, for example, but both of these experiences can be categorized as grapheme-color synesthesia. Ward says ticker taping did have associations with other types of synesthesia, though these associations weren’t as strong as the ones that some other types of synesthesia have with one another. In general, however, “the more types of synesthesia you have, the more likely you are to develop another kind of synesthesia,” Ward says. “It’s almost as if the brain develops synesthesia but not just once. It can develop over and over again.” In Cohen and Hauw’s study of 26 ticker tapers, 69 percent reported that they had at least one other type of synesthesia. (I personally associate a few letters of the alphabet and days of the week with colors but not all of them. Although those pairings are always present, they don’t seem as vivid to me as my subtitles.)

Functional magnetic resonance imaging (fMRI) studies by Cohen, Hauw and their colleagues are starting to provide insight into ticker taping’s neurological basis. In a case study of a single individual with ticker-tape synesthesia, the researchers found that, when listening to speech, certain areas of the brain’s left hemisphere were more active in the ticker taper than they were in control participants. These areas included the inferior frontal gyrus, the supplemental motor area, the supramarginal gyrus and the precuneus, which are all involved in speech processing. Also included was the visual word form area, a region of the cortex thought to be involved in identifying written words and letters. When the researchers had the ticker taper read written words, the same brain areas encompassing both speech and text processing were activated. In other words, the individual’s reading network seemed to be overactivated when listening to speech. The researchers suggest that ticker taping might result from atypical development involving hyperconnectivity between brain areas for speech and vision when people learn to read. Another fMRI study co-authored by Cohen and Hauw that has not yet been published shows similar findings of brain overactivation in 17 ticker tapers.

Further suggestive evidence comes from studies on dyslexia that demonstrate reduced connectivity in these brain areas. Cohen and Hauw have posited that ticker taping could be considered the opposite of dyslexia, although Cohen cautions that this hypothesis may be simplistic. “Dyslexia is a very diverse set of conditions…, and possibly ticker-tape synesthesia may also be relatively diverse,” he says. “I’m not sure to what extent it’s exactly accurate to present both conditions along a single continuum, but that’s the idea.”

“It’s really just speculation because we have not compared both groups,” Hauw says. He notes that more studies are needed to compare the brain activity of ticker tapers with people with dyslexia, allowing researchers to learn more about both conditions. “It can help us to better to have a better understanding of how the brain works and how different regions are connected,” Hauw adds.

In addition to learning more about the brain’s reading network, I’ve come to better understand myself and other ticker tapers through these studies. None of the researchers that I spoke with for this article have this form of synesthesia themselves, and I still haven’t met someone else with it in person—that I know of, at least. But it’s nice to see that enough of us are out there to spur a growing amount of research. This work sets its sights beyond solely studying the seeming quirkiness of ticker taping. Insights about how the brain processes words could illuminate a vast continuum of human experience.


Leave a Comment