Embodied-knowledge technology

Background

We are researching the extraction and sharing of embodied knowledge (i.e., skills) that cannot be acquired through verbal instruction to clarify the mechanism of acquisition of embodied knowledge in sports and establish technology for teaching such skills remotely. Acquiring embodied knowledge means being able to modify and improve how one moves their body (muscles and bones) on the basis of the unique sensations that occur while performing a certain physical action. However, such sensations are only experienced by the person who performed the action, and it is difficult to directly capture and communicate subjective sensations to others.

With our embodied-knowledge technology, we aim to extract and share the unique experiences of professional athletes during high-performance sports by capturing their physical activities as well as the features of the natural environment and tools (i.e., state information) and reproducing these factors to create a similar experience for learners to acquire the embodied knowledge of professionals.

embodied knowledge technology

Approach

As an example of embodied-knowledge technology, we are collaborating with an organization that oversees windsurfing competitions in Japan to develop technology to observe the training of professional athletes, acquire data, discover and share embodied knowledge from that data, and apply it to training to improve athletes’ performance.

Technology for discovering embodied knowledge

In the development of technology for discovering embodied knowledge, we are continuously sensing the behavior of equipment during athletic competitions and extracting the behavior directly related to performance through data analysis and interviews based on subjective evaluation and bird's eye view video. We have developed methods of estimating and comparing joints from video images, extracting target joints, defining the state of focus points, and representing them in languages (Body Map Table). We have also developed methods of analyzing and extracting minute changes that are difficult to capture with visual capabilities from minute color/motion changes in camera images, and present them by highlighting and visualizing them in video. Lastly, to discover embodied knowledge that contributes to performance improvement, we are researching methods of extracting movement features from video using a DNN model and quantitatively evaluating the quality of human body movements by comparing them with ideal movements (Action Quality Assessment).

Technology for sharing bodily sensations

We have begun constructing a simulator that reproduces the state information in a manner that reproduces the unique sensations of windsurfing on the basis of the extracted information. This simulator creates the sensations of by simultaneously presenting windsurfing equipment being used by a professional windsurfer, wind (natural environment), and playing a video of the windsurfer’s perspective. We believe that this simulator will make it possible to share subjective experiences, which are normally difficult to share and limited to the individual. Professional windsurfers are in the process of evaluating this simulator and providing feedback. Through these efforts, we strive to improve the extraction and sharing technologies with the goal of acquiring embodied knowledge to reach targets such as speeds of up to 60 km/h or more in future windsurfing competitions.

Bodily-sensory-reproduction simulator

Future works

We will continue to improve the simulator with the aim of expanding it to non-sports fields which involve physical movements.

Activities

* A list of publications can be found on this page.