Scroll for More
Humans, who have long dreamed of interspecies communication, are now working to decode the calls of two whale species. But our efforts to understand our underwater neighbors require more than just scientific investment—they take real empathy.
One late-summer evening in 2004, a handful of people stood by a lighthouse on an island off the coast of Washington State, watching killer whales.
The group of a dozen or so orcas was heading south, likely bound for a spot near the southern tip of the island that’s prime habitat for the whales’ favorite food: Chinook salmon. So this was a usual sight, a well-worn whale road. But on this evening, one of the orcas broke away from the rest and swam close to the rocky shore.
The whale, a 13-year-old male called L85 or Mystery, raised his head above the water. Wreathed in bull kelp, he called out into the air for several minutes, like an actor on some kind of aquatic proscenium.
“What are you saying?” Monika Wieland Shields, one of the whale watchers on the shore that evening, remembered thinking. Wild killer whales rarely vocalize above water, and they certainly don’t deliver soliloquies. “And what are you saying to us?” After all, Mystery was so close that he must have been aware of the people standing transfixed along the shore.
At the time, Shields was a summer intern at the Whale Museum in Friday Harbor, Washington, charged with listening to and categorizing orca calls she recorded on a hydrophone installed near the lighthouse. She recognized the call Mystery was making over and over that night as S2iii, pronounced “S-two-type-three”: a brief, rising and falling squeal with the emphasis landing on the rise, like a trochaic foot of poetry.
S2iii is the most common of more than a dozen high-pitched, one- to two-second calls made by the subgroup of killer whales to which Mystery belongs. It’s a contact call, Shields said, an ongoing roll call the whales employ to help keep together as they travel and forage for salmon.
The obvious interpretation, then, of Mystery’s message that night: “he was making contact—literally,” Shields, now the director of the Orca Behavior Institute, said.
Scientists and the public alike have been fascinated by the calls, songs, and other sounds whales make since the middle decades of the twentieth century, when it became clear that cetaceans are profoundly vocal creatures. Now, advances in machine-learning and language-translation algorithms may be on the cusp of transforming that fascination into deeper understanding. Research focused on two cetacean species, orcas in the Pacific Northwest and sperm whales in the Caribbean, is testing the potential of computers to interpret whale communication. These efforts depend as much on empathy as they do on artificial intelligence. And each could aid the urgent quest to protect whales from the harm caused by human activities, all while enlarging our sense of what it means to be human and to share the planet with other intelligent beings.
Mystery is part of the Southern Resident orca population, an extended family of killer whales that traditionally spends much of its time in the waters around Seattle, Washington, Vancouver, British Columbia, and the Strait of Juan de Fuca. Different killer whale groups around the world have unique dialects, each group making its own set of calls. And the Southern Residents are known for being chatty even by killer whale standards. “They’re so communicative!” said Ruth Joy, a statistical ecologist at Simon Fraser University in Burnaby, British Columbia. “You listen to the hydrophone when the Southern Residents are going by—it’s just non-stop talking.”
Joy helms the Humans and Algorithms Listening to Orcas, or HALLO Project, one of two efforts to train computers to recognize Southern Resident calls. The idea is to be able to automatically detect the Southern Residents’ presence via a network of hydrophones and to track their location in real time.
The work could have life-or-death stakes for the whales. The Southern Resident orcas are listed as endangered in both the United States and Canada, and their favored waters are crisscrossed by busy international shipping lanes. Knowing when the Southern Residents are present and where they are could lead to alerts telling ships to slow down and quiet their engines, reducing the risk of vessel strikes and making it easier for the whales to navigate and forage for salmon using echolocation.
Finding out what whales are saying could transform our relationship with them even more profoundly, in ways we can’t yet imagine.
Computationally, this is not a simple undertaking. The first task is to teach a machine-learning algorithm what a Southern Resident call is by training it on thousands of different examples of each call type: not just the trochee of S2iii but also the kitten-like mewl of S16, the ascending swoop of S19, and about two dozen others.
Equally important, though, is training the algorithm to recognize what a Southern Resident call is not: the vibrato cheers of Bigg’s killer whales, who inhabit the same area; the low rumble of a territorial male harbor seal; the droning hum of a plainfin midshipman fish looking for a mate; the creaks and clanks of fishing gear; and on and on.
But coding is a straightforward task compared to the physical demands of eavesdropping on underwater cultures, said David Bain, chief scientist at the Orca Conservancy in Seattle, one of the partners in the second Southern Resident-focused effort, known as AI for Orcas. (The two projects are not related, but much like the Southern Residents themselves, they ping each other frequently across the international border.)
“The hard part has been keeping the hydrophones working,” Bain said. A hydrophone cable, like an umbilicus connecting the whales’ sensory world with our own, is easily severed in a storm. Or the hydrophone itself may develop a leak and mask the whales’ calls with a constant buzz. “Electronics and saltwater don’t get along very well together.”
But advanced technology is not the only factor that makes these efforts possible; humans’ connection and careful attention to the whales come into play, too. Joy’s team is working to coordinate automatic detection of vocalizations from hydrophones with visual observations from a network of shore-based volunteer whale watchers. Together, these data will feed into a model that will not only pinpoint where the Southern Residents are but also predict their movements hours into the future, rather like a hurricane prediction system but for beloved weather.
An Ocean of Meaning
While orcas communicate with squeaks and squeals, sperm whales use clicks, intense pulses of sound that are among the loudest sounds made by any animal on the planet and can propagate underwater across tens of miles. Like orcas, different sperm whale groups have distinct vocal dialects defined by Morse-code-like patterns of clicks known as codas.
So far, most research on sperm whale communication has focused on defining the coda repertoires of different communities: this group of sperm whales makes a lot of 5R codas, a series of five clicks at evenly spaced intervals; this other group favors a coda called 1+1+3, consisting of a click, a pause, a click, a pause, and then three clicks in rapid succession. Some codas function as identity markers that indicate a whale’s membership in a certain cultural group, research last year confirmed.
To understand more about what sperm whale clicks and codas mean, a group of about three dozen researchers worldwide have launched the Cetacean Translation Initiative, or Project CETI. The aim is distinct from the orca projects: they hope not just to detect and categorize sperm whale vocalizations but to decode them, too.
To accomplish that, Project CETI researchers plan to record tens of millions or even hundreds of millions of clicks, primarily from sperm whales that are part of the Eastern Caribbean Clan, whose characteristic coda is 1+1+3. Meanwhile, aerial and underwater drones and other noninvasive instruments will record the whales’ behavior to capture how codas are embedded in the patterns of sperm whale daily life.
“I think that the interesting handholds [will be in] social identity, group behavior—in terms of movement and collaborative hunting—and calf rearing,” said Shane Gero, a marine biologist at Carleton University in Ottawa, Canada and Project CETI’s field biology lead. Gero has been studying the Caribbean sperm whales for nearly two decades, gathering detailed observations of mother-calf bonds, friendships, babysitting arrangements, and other family and social relationships among the clan that provide another key underpinning for the project.
Then, the researchers will analyze the sperm whale clicks with machine-learning algorithms similar to those used to analyze and automatically translate human languages. This will detect patterns in the codas: the context in which particular codas are used, the structure of conversations between whales, the minute variations between instances of the same coda, the acoustic properties of the clicks that compose these codas. This breadth will enable the researchers to formulate hypotheses about what the whales are saying. Eventually, researchers want to test those hypotheses by playing codas back to the whales and observing their responses.
After two years of planning, the researchers are now starting to deploy their instruments in the waters off the Caribbean island nation of Dominica. Results of the first analyses might come in late 2023 or early 2024, said David Gruber, a marine biologist at Baruch College at the City University of New York and Project CETI lead.
Decoding sperm whale communication could advance our understanding of how ship noise and other sounds related to human activities in the water affect the whales’ lives and motivate us to minimize those impacts, Gruber said. But the project could also have more far-reaching implications.
The discovery of humpback whale song in the 1960s contributed to the passage of the Marine Mammal Protection Act, which in turn prevented the extinction of several whale species. Finding out what whales are saying could transform our relationship with them even more profoundly, in ways we can’t yet imagine.
It’s a relationship that, especially in the case of the sperm whales that are Project CETI’s focus, has a fraught past. “This is Moby Dick,” Gruber said. “This is an animal that’s been vilified, and its head has been cracked open to light candles—and it could be one of the most intelligent animals on the planet, or in the universe.”
The researchers’ aim now is entirely opposite to that history, morally speaking, yet somehow parallel: the task is to enter into the whales’ minds, ever so gently (Project CETI researchers have vowed never to break the skin of the animals they are studying), to light a candle of cross-species insight.
“Project CETI is about listening to what’s important to the whales and trying to understand what that is and why.”
A Curious Intelligence
Artificial intelligence will be crucial to getting inside the whales’ heads because it gets us out of our own. “We pay attention to certain things because we’re biased as humans,” said Gašper Beguš, a linguist at the University of California, Berkeley and Project CETI linguistics lead. But the algorithms “could pick up on anything that is meaningful.”
It’s tempting for human brains to cast codas as equivalent to words, for example. But there simply aren’t enough different types of codas to convey the infinite shades of meaning that words express. Instead, it could be that there are smaller units of meaning within codas or that variations in the duration or intensity of clicks hold some significance. “Between coda and click, there’s a lot going on,” Beguš said.
Is sperm whale communication language? This, too, may miss the point. Sperm whale lives are so different from our own that “I think we do a disservice to the whales and to humans if we assume that they have all of the same properties of their communication system,” Gero said. Plus, “humans are really good at moving the bar,” constantly redefining what makes us distinct from other animal species.
“But I think that’s the wrong way to look at this,” Gero explained. “For me anyway, Project CETI is about listening to what’s important to the whales and trying to understand what that is and why.”
Researchers are also wrestling with similar questions about how orca calls might encode meaning. For now, machine-learning efforts are focused on simply knowing where Southern Residents are. But within a few years, more and larger arrays of hydrophones in the water and further development of detection algorithms could enable scientists to identify calls made by individual orcas and track underwater conversations, predicts Scott Veirs, a bioacoustician at Beam Reach who helps coordinate Orcasound and has contributed to both AI detection projects focused on orcas. Combining these data with shore-based observations and drone research—a kind of grassroots, ad-hoc version of the Project CETI approach—“the stage will be set for understanding how [orcas] communicate,” Veirs said.
Some researchers suspect orca calls are rather simple or general in meaning. “I think more of the information is in the context,” Bain said. That is, a call of excitement made when a loud boat is passing nearby would have a different meaning than the same call made when meeting a new baby in a neighboring pod.
Others believe the calls have unplumbed depths, with minute variations in frequency, amplitude, and harmonics carrying the nuances. “What we call an S1 might be a hundred or a thousand different things to them, depending on who’s saying it and how they’re saying it,” Shields said.
One evening early last December, a hydrophone newly installed off Vashon Island near Seattle picked up sounds made by J pod, one of the three pods that make up the Southern Resident community. Over the next several days, Veirs pored over the recording, zooming in on parts of the spectrogram where likely orca vocalizations were indicated by clusters of spikes, a visual echo of the fins of a killer whale pod at the surface of the water.
“All I’m hearing are S4s,” Veirs said, referring to a Southern Resident call that sounds to human ears like a goose honk. That’s unexpected because J pod’s signature call is S1, a long squeal that drops in pitch at the end. But the three-hour recording contains no S1s at all.
Is this pattern part of a larger shift in J pod’s repertoire? Does it reflect the acoustic environment, perhaps the presence of more ship noise? Or were the whales having a conversation on a topic different than what they would talk about with S1s?
In addition to calls, orcas also make clicks and ultrasonic whistles, and Veirs and other researchers have recently become more interested in these vocalizations, hypothesizing that they, too, may convey meaning. “I’m just beginning to propose some nomenclature for categorizing clicks,” Veirs said. “But we’re at the point now where I’m labeling these as ‘cool click sequences,’” rather than official, scientifically agreed-upon names.
The spectrogram from that evening is dense with Veirs’s annotations: “clicks, slow, overlap;” “S04, faint;” “whistle, upsweep;” “cool whistle + click sequence.” Something about the tentative, exploratory nature of those labels recalls Mystery and his long-ago vocalizations into the unfamiliar medium of the air: the record of a curious intelligence encountering something possibly significant and definitely wondrous, but not yet knowing quite what it means.
Music Pour Le Sport @ 11th House Agency Sound Recordings Monika Wieland Shields, Lime Kiln Hydrophone, The Whale Museum; Scott Veirs, Orcasound; The Dominica Sperm Whale Project
This article first appeared in Atmos Volume 08: Rhythm with the headline “Sound Waves.”
Nature is an elaborate orchestra of interconnectedness, in which timing is everything.