Smart & Quick Extraction of ObjectsObject extraction technologies using deep learningNTT Human Informatics Laboratories

By learning the object using deep learning, it is possible to flexibly extract objects with various appearances in various environments. This can extract the desired object accurately and in real time from not only simple scenes such as live presentations, but also from scenes that have changing lighting conditions such as performing arts.

The object extraction technology used to separate desired objects in videos may be applied to content creation and video communication, and there are a number of previous studies in this area. NTT have been researching and developing communication technology capable of providing high sense of realism, as if performers were right in front of the viewer, and this uses object extraction as a core technology. This technology is intended for use in entertainment fields such as performing arts, enterprise fields such as live presentations, and sports fields. Since it is extremely difficult to extract only the object accurately and in real time from the video shot under any background, it is common to shoot using a single color background (green screen, etc.). However this technology makes it possible by learning the object to be extracted in advance by deep learning.

• Improved robustness against changes in the background environment using deep learning, and highly accurate extraction of boundary areas.
• Proprietary algorithm enables real-time extraction of the object area for Full HD or 4K video.

• By extracting only the actor from the stage video and compositing it with other live video or CG, it can be used for new video expression on the stage.
• Extracting only the athletes from the video of a sports match and projecting these on a stage capable of life-size reproduction lets us share the excitement of the venue.
• In video production sites such as broadcasting and entertainment, by applying this technology to video shot in places where it is difficult to set up a green screen, it makes it easy to create a video that extracts the object area.
• Extracting only the speaker from scenes such as lectures or keynote speeches, and projecting on the screen of another venue, the quality of the viewing experience at a remote location is improved.

NTT Human Informatics Laboratories - Cyber-World Laboratory