The development of IoT and AI technologies has driven an increase in the volume of data exchanged across the world. However, this has brought a number of problems, such as increases in the cost of data communication and processing, and the challenge of ensuring safety. Today, we will be taking a look at next-generation data hub technology designed to solve these problems.

What Is Next-Generation Data Hub Technology?

Advances in IoT technology have resulted in data being collected and sent to servers by smartphones, a variety of new types of sensor, surveillance cameras, and more. The development of AI technology has also given rise to a number of services that process the data collected in this way at high speed on servers and thereby increase the value of data.
As a result, the amount of data exchanged around the world continues to increase, and this trend is expected to continue. However, we cannot ignore the fact that the increase in cost due to the increase in the amount of data and the challenge of finding a means to protect confidential data for service delivery are hindering the spread of services.
Next-generation data hub technology is expected to overcome these challenges and create a truly data-centric society.

In traditional data-leveraging services, all the data acquired from sensors is sent to the service provider's server, as shown in the left-hand side of Figure 1. The data is then processed, and the service is provided based on the processing results. As the number of sensors and services increases, this system increases both the amount of data transferred and the amount of data processed.

On the other hand, the next-generation data hub on the right-hand side of Figure 1 only sends the necessary data to the server for processing, based on its content. The next-generation data hub enables mutual use of AI-analyzed data, reduced data-transfer costs through metadata-based delivery, more advanced data access, and secure usage and leverage of data. For example, we may reach a point where we are able to provide services that solve societal issues, such as real-time hazard prediction, and support of individuals needing assistance, on a large, city-wide scale.
Next, we will take a look at three distinctive features of next-generation data hub technology:

Next-Generation Data Hub Technology Features (1) — Event-Driven AI Inference

The first is event-driven AI inference.
This technology inspects data captured by sensors and sends the server only the necessary events which trigger subsequent deeper inference tasks, rather than sending unfiltered data and the like. Data inspection such as object and motion detection is performed on the device (edge) side, and only inference requiring advanced processing, such as risk detection and prediction, is carried out server-side.

Taking image processing as an example, only the parts of the image that feature a human—more specifically, the skeletal and facial information— are sent for processing on the server side. This results in a reduction of the volume of data transferred and processed.

Next-Generation Data Hub Technology Features (2) — Data Flow Control

The second feature for us to take a look at is data flow control.
In a data-centric society, one piece of data is processed in a variety of ways and used by multiple services. Data flow control technology is designed to offer flexible and efficient data processing by standardizing a data acquisition method across multiple services, replicating data on the server side and performing common pre-processing on the data before it is shared, as appropriate.

Next-Generation Data Hub Technology Features (3) - Data Sandbox

The third feature is data sandbox.
It is difficult to circulate confidential information safely using the currently existing system. To address this, the next-generation data hub provides "data sandbox" environments that are isolated from third parties for safe data sharing and processing.
Specifically, when carrying out processes such as statistical analysis, learning, and forecasting, data and algorithms are only decrypted within the CPU to preserve confidentiality. The sandbox also hides analytics algorithms placed within the data sandbox, and leverages technologies such as mutual authentication and key exchange to allow data and algorithms to be shared and utilized between large numbers of people.

Outlook for the Future

That concludes the introduction to our next-generation data hub technology, which reduces data transfer and processing costs while safely connecting data owners with data users.
NTT plans to continue its research into next-generation data hub technology with its partners in the IOWN Global Forum and elsewhere, with the goal of building a data-centric society.