Motor-skill-transfer technology

Background

Our bodies move through a process in which motor signals are transmitted from the brain to the muscles and cause the muscles to contract. The brain then perceives and recognizes the results of the movement as stimuli through sensory organs and repeats planning and giving new movement instructions. The combination of sensing and feedback locations, as well as the specific sensing and feedback methods, changes the content of motion support. Motor-skill-transfer technology supports communication and movement control in the human body through analysis and electrical-stimulation feedback on the basis of the sensing of biological signals such as brain waves and muscle potentials. We have begun investigating the following two areas: muscle activity measurement (via electromyography), motor status sensing, and motor support (intervention) using electrical muscle stimulation (EMS) based on the motor activity; and sensing and　intervention regarding perception in the brain in response to input from sensory organs. We have also started to focus on vision, somatosensation, and vestibular sensation, which are important in postural control and serve as the basis of movement.

In physical education or personal training, the instructor and students are typically present in the same space , and the instructor provides instruction through words and gestures. As an alternative to in-person instruction, we aim to develop ways of transferring motor skills through biometric sensing and intervention to enable instruction regardless of time and location and to create a space (Remote World) that provides more in-depth instruction than in-person classes. As an example of training in the Remote World, we describe the remote instruction of two kinds of piano techniques.

Assistance techniques for tremolo on the piano

Beginners and experts use their arm muscles differently when playing a tremolo on the piano; that is, beginners focus on finger movement, while experts focus on wrist rotation. Focusing on this difference in muscle activity, we are developing a technology that uses EMS to directly transfer the way an expert uses their muscles to the muscles of a beginner. Through experiments using our developed technique, we determined that the beginner can directly learn how skilled pianists efficiently move their bodies in a manner that enables them to play with less unnecessary effort in their forearms.

Technologies to teach usage of muscles in playing C major scale on the piano

Scales, in which a sequence of notes are played in order, are a fundamental skill in piano playing. The skill is evaluated quantitatively by the evenness of the tempo and duration of each note. We conducted a pilot study in which experts and beginners were instructed to play the C major scale, and the experts played more evenly than the beginners in terms of both speed and note duration. In particular, the duration of the E note was longer for the beginners. The differences were due to the beginners’ lack of using the thumb-under method, which is to pass the thumb under the palm before pressing the F key with the thumb. From the analysis of the thumb-under method, we found that the experts used the flexor digitorum muscle on the forearm to press the keys and the deltoid muscle on the shoulder to slide the hand sideways. Thus, we have developed an EMS-based support system for beginners to learn the appropriate muscle synergies of the forearm and shoulder muscles to improve their technique when playing scales on the piano. Through a user study, we found that the evenness of the speed and duration of each note improved after practicing with EMS. In particular, practicing with EMS was more effective than without EMS in improving the evenness of speed.

Application

With this technology, we developed a conceptual system for training in the Remote World. A simple motion sensor is attached to an expert pianist's forearm to measure the rotational motion of the wrist and transmit it to the system. An EMS device is connected to the arm of the learner, and the EMS is applied to alternate rotations of the wrists as the expert plays.

Experienced players can be taught through audio and video as well as by directly experiencing the playing techniques of expert pianists through the system. This creates a new teaching and learning experience for both the instructor and student. We are focusing on the coordinated movement of multiple muscles during a performance and investigating a system that allows multiple muscles to move in coordination through EMS.

Future works

By applying the various sensing and intervention technologies we are developing for our motor-skill-transfer technology, we will continue to create new forms of education and training in the Remote-World era for specific cases such as everyday movements, sports, and playing musical instrument.

Activities

* A list of publications can be found on this page.