03/08/2022

    Venue-management Staff × CUzoNTT Human Informatics Laboratories

    Venue-management Staff × CUzo

    • CUzo
    • transparent displays
    • AR

    For the 2021 international sporting event in Tokyo, NTT brought out a feature-distribution communication technology called CUzo and a mobile device with a transparent display called CUzo Card. This article describes our work on this project, which sought to enhance the hospitality afforded at venues by providing venue staff with information and communication technology tools to support the guidance they provide to visitors, and the technology used in this project.

    Takahiro Kusabuka, Yuichi Maki, Takuya Gouda,Akira Suzuki, Masaya Tamaru, and Takuya Indo
    NTT Human Informatics Laboratories

    Overview

    The purpose of this project was to improve the hospitality provided at venues by using information and communication technology (ICT) to support the efforts of venue-management staff who provide guidance to visitors including athletes and spectators. It is commonplace for people to use smartphones and tablets when dealing with inquiries, but they tend to do so while constantly looking at the screen of their device. This makes it difficult to properly engage in communication with other people while understanding their intentions based on non-verbal cues such as facial expressions and gestures. For this project, we used a system consisting of a feature-distribution communication technology called CUzo and handheld device called CUzo Card with a transparent display based on this technology (Fig. 1, Table 1) with which we implemented a communication style whereby venue-management staff could provide visitors with guidance while observing visitor expressions.

    Fig. 1. A CUzo Card.
    Table 1. CUzo Card specifications.
    Fig. 2. Using the face-to-face translation app.
    Fig. 3. Using the facility guidance app.

    Applications

    Facility guidance

    When venue-management staff are showing visitors the way to their seats and on-site facilities, they can provide clear guidance by holding up the CUzo Card to provide annotations on the visitor’s actual surroundings, including the location of the facility and the route to their destination.
    Previously, it would have been necessary to go through the complicated procedure of searching for the target facility on the basis of the information confirmed from a flat map while comparing this information with the actual scenery then rebuilding a mental picture of the route. Some venues do not have signs or facilities that can serve as landmarks, and even if explanations are attempted using descriptive words such as “the white sign,” this can still lead to difficulties where smooth guidance is not possible when both parties understand these words as referring to different landmarks.
    CUzo can solve this problem by superimposing annotations on the user’s actual surroundings so that they can be seamlessly guided through the facility. In this project, we prepared guides to show people to their seats at sports events, and to other facilities such as restaurants, stores, toilets, multi-purpose (barrier-free) toilets, prayer rooms, lounges, medical offices, and automated external defibrillators.

    Face-to-face translation

    Although smartphone translation functions can be used to handle multiple languages, these functions are not geared toward face-to-face conversations, but it is assumed that both parties in a conversation are speaking while checking their smartphones for the translated results. As a result, they end up being glued their phones and more likely overlook the other person’s facial expressions and gestures, which are regarded in conventional customer service as being important for understanding what someone is really trying to say. To address this issue, we designed an interactive system that allows users to view information while looking at the other person’s facial expressions through a transparent display. This system is used by holding up the display between the two participants in a conversation. When one of them speaks, their words are translated into subtitles that are displayed where the other person can see the speaker’s facial expressions and gestures together with the content of their speech (Fig. 4(a)). This system supported eight languages: Japanese, English, Spanish, French, Portuguese, Chinese (traditional and simplified), and Korean.
    Due to the COVID-19 pandemic, it was felt that people might be unwilling to engage in face-to-face conversations, so we prepared the screen design shown in Fig. 4(b) to enable people to converse in a side-to-side manner without having to flip the screen around. Users can switch between the two screen designs at the press of a button, allowing the system to respond flexibly.

    Fig. 4. Screenshot of the face-to-face translation app.

    CUzo feature-distribution communication technology and the CUzo Card

    The CUzo feature-distribution communication technology allows high-performance services to be accessed through simple devices by using communication to link up with distributed functions deployed on a network. By keeping the devices simple, we were able to address issues that can degrade the user experience, such as lack of responsivity when the device is subject to increased processing loads, and we were also able to improve the device’s battery lifetime and reduce initial deployment cost. The CUzo Card is a compact and lightweight device based on CUzo, incorporating the latest transparent display technology. Since the CUzo Card has a transparent display, users can view information from both sides of the display while looking through it at their actual surroundings.

    System

    The system we provided for this project is shown in Fig. 5. Venue-management staff carry CUzo Cards on neck straps and can assist visitors by communicating with them via facility guidance and face-to-face translation apps. The app server runs on the cloud, and the CUzo Card connects to this server via a smartphone over a Bluetooth® connection.

    Fig. 5. System configuration.

    Results

    Since the international sporting event went ahead in 2021 without any spectators, only the face-to-face translation function was used. However, the CUzo Card was widely used for guiding foreign athletes and staff. Over a 28-day period, approximately 3500 translations were carried out by approximately 250 venue-management staff members at three venues. The system received favorable responses from users, who not only thought it looked cool but also appreciated being able to see who they were speaking to. This was due to the attractive and convenient design of the device based around a transparent display, providing users with a tool that allowed them to converse more naturally with other people while observing their expressions.
    Users also found the device easy to operate and use, confirming the utility of our user-experience design of enabling people to use the system without disrupting actual work situations. However, some venue-management staff members who used the system in direct sunlight said that it was sometimes difficult to see the screen, so we will need to consider the characteristics of the display when used outdoors.
    Regarding the translation function, some users said they were glad to be able to access other language pairs besides Japanese and English, and the system was welcomed by users who had few staff members capable of speaking languages such as French or Spanish. It was also pointed out that the speech-recognition results were sometimes inaccurate, and that the translated results sometimes took quite a while to appear, highlighting the need for improvements to the speech-recognition/translation engine performance and the bandwidth of the local network through which the devices were connected.

    Summary

    By applying our CUzo feature-distribution communication technology and our CUzo Card device with a transparent display for providing guidance for visitors to venues used during the 2021 international sporting event, we were able to provide a system that allows this sort of work to be performed more smoothly. This system is expected to be useful in a wide variety of customer-facing scenarios. Examples include public transport facilities such as train stations and airports, public amenities, and tourist facilities. Although in this case we asked the venue-management staff to use the system, the fact that they found it so easy to use suggests that CUzo Card could also be carried by visitors and tourists while travelling or when visiting places such as art galleries and museums.
    In the future, we intend to continue with our ICT research and development to explore new experiences at places where people and information come into contact.

    Takahiro Kusabuka, Yuichi Maki, Takuya Gouda,
Akira Suzuki, Masaya Tamaru, and Takuya Indo

    Takahiro Kusabuka, Yuichi Maki, Takuya Gouda, Akira Suzuki, Masaya Tamaru, and Takuya Indo

    Related content