An AI started tasting colours and shapes
The brain often blurs the senses – a fact that marketers often use in the design of food packaging. And AIs appear to do the same.
What is the flavour of a pink sphere? And what is the sound of a Sauvignon Blanc?
Such questions may sound ridiculous, but a huge body of literature shows us that the human brain naturally merges sensory experiences. We may not be conscious of the phenomenon, but we associate different colours, shapes and sounds with different flavours in ways that can subtly shape our perceptual experience, for example.
The colour of our glass, or music playing in the background of a bar, can determine how sweet or musky a wine tastes, for instance. "This cross talk between the senses is happening almost on an ongoing basis all the time," explains Carlos Velasco at the BI Norwegian Business School in Oslo, Norway. In extreme cases it can manifest in a blurred sensory experience for some people where words might trigger specific tastes or music produces a riot of colour – something known as synaesthesia.
And while the idea that you can "taste" a colour or sound may seem absurd enough, Velasco's latest research suggests that generative artificial intelligence systems may also do this too. As with all AI algorithms, this is largely a reflection of biases in the data they were trained on, so they are perhaps just highlighting how common these associations may actually be. But Velasco and his colleagues hope to use this fact to find many other ways to hack human senses.
Eating with the eyes
First, a note on terminology. Scientists use the term "sensory modality" to describe the means that the body uses to encode information – through, for example, our taste buds, ear drums, the retina in our eyes or the "tactile corpuscles" in our skin. The associations that we tend to form between different sensory qualities are therefore known as "cross-modal correspondences".
Experimental evidence for this phenomenon first emerged in the 1970s, with studies suggesting that red and pink hues are associated with sweetness, yellow or green with sourness, white with saltiness and brown or black with bitterness. These general patterns have now been replicated many times since, using multiple experimental methods.
Participants may be asked for their subjective judgement of abstract questions such as: "On a scale from 1 to 10, with 10 being the most sweet, how sweet is the colour red?" From this, you can see that, on average, each colour has a unique flavour profile shared by large numbers of people across different cultures. A multinational collaboration, led by Xiaoang Wang at Tsinghua University in China, found similar cross-modal correspondences in Chinese, Indian, and Malaysian participants.
Alternatively, participants may be given a particular food or drink presented in multiple colours, and asked to judge the taste of each one. Eriko Sugimori and Yayoi Kawasaki at Waseda University in Japan, for instance, have found that bitter chocolate tastes considerably sweeter when it is wrapped in pink, rather than black, packaging.
1034
