Sharing sports with the blind: Exploring new ways to watch sports through sense of touchNTT Service Evolution Laboratories
NTT Communication Science Laboratories

2020 Epoch-making Project of NTT Service Evolution Laboratories started a new-perspective sports study in 2018 to allow people with disabilities to enjoy sports through haptic sensation. In addition to research work, the project's ambition to share a haptic sports experience and re-examine the essence of sports includes operation of the "Sports Guide without Sight" website and a book and exhibitions at the NTT Inter-Communication Center (ICC). We discussed these expressions and concepts, recent content development, and future plans, with Junji Watanabe of NTT Communication Science Laboratories, who works in the field of cognitive science and publishes FURUE magazine dedicated to haptic content, and Akiko Hayashi of NTT Service Evolution Laboratories, who researches user experience (UX) design.

In the past, haptic research was focused on personal communication and entertainment, but sports was not a topic of interest. However, after Japan was selected to host the 2020 Olympic and Paralympic Games, a mission was born to provide new services for this. The Tokyo 2020 Games has three basic concepts, one of which is "Diversity and Inclusion." With "diversity" as a key word, we decided to work with associate professor Asa Ito of Tokyo Institute of Technology, and people with vision loss, to figure out a way for an audience with visual impairments to enjoy watching sports. Sports-related research is often focused on the athletes, but in a broad sense, sports involve both athletes and spectators in a unified event. Watching sports is more fun when you can share the experience with people around you and relate to what's happening in the game. In these events, spectators are the obvious majority, and athletes are a minority. So, we thought it would be worthwhile to explore the experiences happening on the audience side.

This project uses existing haptic research tech to bring people together through shared experiences by proposing new ways to watch sports and exploring ways to enjoy watching sports together with people who have disabilities.

What can we do to help the blind enjoy sports? When you watch a game, it's common to describe the situation in words, but even if you explain what you're seeing in detail, there is a limit to how well the intensity, timing, and detail can be conveyed, so it’s not easy to foster a sense of togetherness with someone who can’t see what’s going on. It is especially difficult for people who are born blind and have never watched a particular sport to imagine what is happening in the game. We began our research with a strong desire to enhance audience experience, but we became increasingly interested in finding ways to enjoy games together with the visually impaired. That's why we've created the Sports Social View method that physically expresses what's happening in a game and shares it with people who have vision loss. This was based on the Social View method that allows us to enjoy art together with the visually impaired to discover new ways to view and interpret art through discussion.

At first, we started our research with an experiment in collaboration with people with visual impairments, where we used a table to mimic a tennis court, set their hand on top of it, and tapped the table to match the rhythm and position of the ball so they could feel it. This provides a sense of the rhythm of tennis and the atmosphere of the rally. We realized that something was missing when communicating sports to the visually impaired, and to express it, we needed to take the essence of sports and re-capture it again, or in other words, “translate Eit. Soccer and rugby, for example, both involve a large playing field and a ball. But thinking about what's different when watching these sports, I would say that the excitement in soccer lies in its overhead tactical strategy, while rugby's appeal is the struggle and strength needed to get the ball, and the powerful impacts involved. Although both sports involve competing for the ball, looking at their essences, they are quite different. We thought it might be possible to re-classify sports from a different perspective such as dynamism of movement, and that it was necessary to extract and translate the essence of the sport in a physical sense. We hoped our translation would allow people with visual impairments to feel that they are part of the events that are happening during the game so they can share the experience.

This project is advancing on two pillars: research, and technology-driven experiential prototyping. The research involves asking sports experts to explain each game while abstracting their responses and translating them into experiences. Specifically, this translation work involves extracting the essence of each sport. For example, in judo, opposing forces are considered to be the essence. This can be translated into two people grabbing each other by the collar and applying force, basically "cloth pulling." In table tennis, impact speed and intensity are very important, but we learned from one expert that ball rotation is also essential. We did a lot of trial and error to explore how to convey a feeling similar to that of rotation, such as hitting a pot lid with slippers. Lectures by experts and experiments using everyday household items are published on the Sports Guide without Sight website (http://mienaisports.com). Recapturing the essence of sports through "viewing with the body," we have showcased the translation of ten competitions: such as rugby, table tennis and sailing.

Sports Guide without Sight involves analog-based translation. Our other pillar is technology-driven experiential prototyping. We exhibited some findings at the "ICC Kids Program 2019: Sports Laboratory" held during the summer of 2019 at NTT InterCommunication Center [ICC] (http://www.ntticc.or.jp/ja) in Hatsudai, Tokyo. We did more than one activity. Our exhibit recreated a subtractive experience of tennis where we mounted vibrating speakers under a table. The table was set up in a small room with vibrating speakers and a large screen. At first, you experienced the vibration of the rally plus video (with sight and hearing). Next, with sound and vibration but without video (subtraction of sight). Finally, only vibrations in complete darkness (subtraction of sight and hearing). This allowed visitors to experience sports from a differently-abled perspective. As a simplified version of this, we also had a small portable-sized exhibit that allowed visitors to experience the vibrations while watching tennis using a smartphone.

For our "Sensing an Invisible Skateboard," we made a full-size ramp (a half-pipe replica) that combined haptic vibrations with footage of skaters moving back and forth on the ramp. You could physically feel the movements (tricks) of the athletes through the ramp vibrations when you touched or lay down on the ramp. Technology is critical to providing more people with new ways to enjoy and watch sports. We want to continue thinking about sports-watching experiences like these using ICT technology.

Our project is fundamentally about organizing, expanding and sharing sports from a new perspective for the year 2020. It includes a haptic and experiential approach to pursue how people with different abilities around the world watch sports together. By changing the channels of our senses, we will be able to offer a different and more engaging sports experience. We believe there is a place for our work beyond the field of sports. Society is comprised of many different people with physical differences and diverse cultures. Ideally, we want our work to communicate how diverse people can work together, what our attitudes should be, and what a symbiotic community should be. Then we'll keep moving forward with our research.

Right now, our project is looking to further explore the essence of other sports that still need more research, like tennis and judo. With more exhibition opportunities like the ones seen at ICC, I believe we will be able to convey our message to a wider audience. In our haptic research, we want to think about how haptics can play a role in the sports world, and how we can create social value through this project.

This project is not so much about providing support, but rather about how we can be made aware of physical differences and appreciate them. A lot of information enters through our eyes, and we understand it unconsciously, but this isn’t the case for people with visual disabilities. However, by thinking about how we can convert the feeling of hitting a ball or exerting force, my perspective changed completely, and I realized how this could lead to new and fascinating discoveries with sports. It’s fascinating to see how re-classifying sports from these perspectives can lead to a system completely different from the traditional game and martial arts categories. And it will be even more fun when our society will be able to share the sports experience with everyone.

Ms. Hayashi, who was originally involved with universal design, and Dr. Watanabe, who studies the connection between well-being and haptic research, are from different laboratories, and their work involves a diverse team including external researchers. Their project started with a goal of completion by 2020, but we hope it will expand and become widely recognized.

Interview by Kanako Kaisho
on July 26, 2019

• Furue VOL. 20, a journal dedicated to haptic studies
  http://furue.ilab.ntt.co.jp/book/201812/contents2.html
• Sports Guide without Sight
  https://www.mienaisports.com/
  https://www.amazon.co.jp/dp/4794971923/?language=en_US