Development of Anonymously processed information creation software which accelerates businesses utilizing personal dataNTT Secure Platform Laboratories

Anonymization is becoming one of the key technologies to utilize big data. The simple definition of anonymization is the process of making personal data into anonymized data from which individuals cannot be identified. Anonymization is used to create “anonymously processed information” that can be used without individuals’ consent. However, it is said to be difficult to make properly anonymized data in the modern information society because there still remains a risk that individuals can be identified even after deleting distinctive data such as names, addresses, ages, and so on. Collecting various data, such as purchase history or GPS data, makes it easier to identify individuals. However, the value of anonymized data will decrease if too much information is removed from it. How can we prevent the identification of individuals without compromising the value of use? To tackle this problem, Anonymously Processed Information Creation Software 2.0 has been developed. This software eliminates the risk of privacy breaches while providing useful information for analysis. We took an in-depth look at the features of this software in the context of their development background.

To use someone's personal information for other purposes like marketing, we must obtain individual's consent regarding the purpose of use. To utilize data for a new purpose, we have to obtain consent again, and it takes enormous effort in terms of time and cost. However, in May 2017, the situation changed significantly due to the amended Act on the Protection of Personal Information. It stipulated that anonymously processed information adequately anonymized can be used for unspecified purposes or provided to a third party without the individual's consent, and the creation of anonymously processed information is attracting attention to enable new businesses.

The anonymously processed information is defined as processed personal information so that specific individuals cannot be identified and original personal information cannot be restored from it. Examples of processing include the deletion of information such as names that can be used to identify individuals directly, the generalization of information like changing addresses from detail to a municipal level, and randomization, which adds noise to the original value or replacing it with another value. When creating anonymously processed information, it must comply with the commission regulations number 1 to 5 below and process data appropriately in consideration of risks each time.

• 1. Deletion of descriptions that can identify specific individuals
  Name, address, gender, date of birth, among other things.
• 2. Deletion of personal identification codes
  Government-issued ID numbers such as Individual Numbers or driver's license numbers and biometric data such as genome data, fingerprint data
• 3. Deletion of linkable codes
  ID codes such as employee IDs or point card IDs issued by the private sector
• 4. Deletion of unique descriptions
  Descriptions that can be unique according to common wisdom
  Ex: Purchase history record of a specific item which is the only one in the world
• 5. Appropriate process according to the property of the database
  Descriptions that can be used to identify individuals even if regulations number 1 through 4 are satisfied

Although anonymously processed information is expected to be a better approach to utilize personal information, there are also some challenges. As described above, it is required to satisfy regulations number 1 to 5 when creating anonymously processed information. Still, there is no quantitative standard for adequately creating anonymously processed information, so the decision is finally left to the data handler (data processor). The next challenge is the usability of anonymized information. Much more processing is required to improve anonymity. However, as much as the information is processed to ensure anonymity, the farther it strays from the original data and becomes less valuable. Usually, anonymity and usability have a trade-off relationship, and it isn't easy to keep both highly balanced.

Even before the amendment of the act, We have been conducting research and development of privacy protection technology for about ten years, realizing the necessity to process personal information, not to identify individuals. Using our technologies and insights, we have developed software that implements a wide variety of processing and evaluation methods, including NTT's proprietary techniques. Anonymously Processed Information Creation Software 2.0 meets the needs of both data processors who want maximum possible anonymity and information users who require information that can withstand analysis.

Lifestyle diversification has caused demands for products and services tailored to customer's hobbies and preferences. Customers also have to access vast amounts of information online, and it isn't easy to find information that suits them. That is why the utilization of personal data is attracting attention. For example, it is expected to produce great results in marketing, such as sending highly accurate recommendations by performing detailed segment analysis of the user group from their purchase history. But privacy protection is a critical issue, and such information is difficult to handle without security. With Anonymously Processed Information Creation Software 2.0, personal information can be processed into usable anonymized information while maintaining a high degree of privacy protection, without losing its value as analytical data. It is necessary to adjust the degree of processing in order to create anonymously processed information that meets with needs of information users. This software provides several functions, processes not only data but also evaluates processed data. If processed data does not match the purpose of use of the data, further processing can be performed, enabling efficient trial and error of processing and evaluation.

One of the significant features of this software is NTT's unique anonymization technique, Pk-anonymization. In comparison to the conventional anonymization technique "k-anonymization," a reduction in the usability of data can be prevented less, and it is possible to create sufficiently high valuable data despite repeated data processing and evaluation. Conventional k-anonymization is a technique that processes data to satisfy a particular safety parameter, called k-anonymity, and that reduces the risk of individual identification to 1/k or less. For example, if k is set as 3, processed information can only indicate three or more people, and individuals cannot be specified. The higher the k value, the more anonymity the data has.

However, simple application of k-anonymization to data may reduce the value of information to satisfy anonymity. For example, although k is set as 5, the more attributes the data has, the re-identification becomes much easy. When trying to create safe data that satisfy k-anonymity, there are some cases where many data, including target data of analysis, have to be deleted. It makes processed data less usable. Pk-anonymization was born from research and development to solve these problems.

Pk-anonymization consists of two processes: the randomization that changes portions of the data in a probabilistic way, and the other is the reconstruction that estimates the original distribution of data from the statistical property. This technique creates anonymized data with high usability that satisfies k-anonymity theoretically while still maintaining the original data's statistical property. It was known that probabilistic randomization of data increases anonymity, but there was no established way how much noise is necessary to put in for satisfying certain anonymity. NTT has invented how to insert the minimum amount of noise that can ensure the same level of security as k-anonymity and succeeded in mathematically proof of its safety for the first in the world.

When using this software, the necessary parameter for adding noise that satisfies safety parameter k, such as k=2, is provided. You can randomize the data according to this parameter and obtain anonymized data that preserves usability without excessive generalization or deletion of the information.

As a side note, NTT recommends using not only Pk-anonymization but also k-anonymization in situations. Some people want to analyze only correct data, although some data gets deleted. In contrast, others care much about data volume, although it has a little noise. Our software allows users to select anonymization techniques according to the purpose of analysis that means how they plan to use the information.

There are several companies that are developing data anonymization software in Japan and around the world, which is attracting attention to enable new businesses. Most of them focus on only processing techniques to make data safe, but we aim to create anonymously processed information that is both safe and usable, keeping security in mind. In version 1.0 of Anonymously Processed Information Creation Software, we have provided the necessary functions for creating anonymously processed information. Furthermore, we have achieved other sophisticated functions in version 2.0. Here we introduce some features of our software other than Pk-anonymization.

It provides a total of 35 types of processing techniques, covering all anonymization techniques described in the Guidelines on the Act on the Protection of Personal Information (Anonymously Processed Information).

• Historical data processing
  For example, in a case, there are some records of Mr. A, whose age is the 30s, in the historical data. When using k-anonymization for 10-year increments, it seems that there are multiple people at the age of 30s in the anonymized data even if there is only one person, because Mr. A repeatedly appears in the data. On the other hand, even it seems safe at a glance, there is a risk that the anonymized records of age of 30s can be linked to Mr. A from original data or others because those records belong to only one person. Therefore, we compress the data to each individual's level by internal processing to be processed appropriately even if the same person appears multiple times.
• Sequential date processing
  This newly implemented technique corresponds to the need for analysis in a time-series order. For example, it was challenging to anonymize date information in medical records such as treatment dates, cure date, the first visiting date, the last visiting date, among other things. Processing the data to secure and usable data required for analysis was difficult for existing conventional software because they needed to delete or generalize too much to keep anonymity. Hence, in version 2.0, we have developed a sequential date processing technique that inserts pseudo data without compromising the sequential order.
• Data adaptive processing technique (processing filter)
  A new filter function is adopted to reduce the processing effort. Thinking about the categorical generalization of heights data, for example, it was necessary to separate the data into the males and females and process separately for generalizing males in units of 5 cm and females in units of 10 cm. However, in version 2.0, it becomes possible to process both data at the same time.

By the way, NTT Laboratories has demonstrated its technological capabilities by becoming the top company for three consecutive years in the anonymization and re-identification contest named Privacy Work Shop Cup (PWS Cup) held by academic societies. We were one of the participants in this contest at that time. Still, we recently participate as a leading member in the secretariat office to contribute to the implementation and raise awareness of anonymization in domestic research. These activities give us the confidence to adopt the best techniques necessary for creating anonymously processed information.

This software provides an easy-to-use graphical user interface for performing processing techniques and displaying assessment results. It supports the efficient creation of anonymously processed information through easy-to-read and easy-to-understand operations, such as the ability to check the balance between anonymity and usability in a graph.

In the business scene of anonymously processed information, we recognize some need for connectivity with external systems, not only using it as a standalone system. We have implemented a connective API for embedding into external systems.

When processing personal information in business, public institutions may ask for evidence that proves the processes and techniques were adequate for data in the case of trouble. Therefore, we have adopted an evidence log output function that allows users to track the anonymization and evaluation processes and a report output function that outputs a summary of the total processing in a report.

NTT TechnoCross has released the commercial version of Anonymously Processed Information Creation Software 2.0 (see https://www.ntt-tx.co.jp/products/anontool/). There are many inquiries, especially from the healthcare and financial industries. Also, the number of inquiries from local governments increases because of the movement to open the data managed by local governments appropriately to the private sectors and promote economic development.

However, utilizing personal data is a kind of new business, so the knowledge and skills to use processing and evaluation techniques adequately are not yet widespread. In order to correspond requests to add new techniques, in version 2.0, we have developed and adopted an interface that allows new techniques to be added so that users of this software can create their processing techniques. Furthermore, as processing techniques has increased various and extensive, users want to be informed which techniques are appropriate and how to use them on their data. Therefore, we are trying to research and development of intuitive technology that unnecessary special knowledge when using.

Actually speaking, even if individuals cannot be identified from the anonymously processed information created today, it might become possible to identify individuals by advanced re-identification technology after 5 to 10 years. We will keep evaluating the practicality of our techniques and improving the software's quality and functionality in commercial use, in correspondence with future revisions of the law and further needs of international markets.

While listening to their story, I have realized that it is a complicated matter to process personal data to protect privacy while maintaining its value as much as possible because the handling of personal information is sensitive. I think that the theory behind NTT's proprietary technology, Pk-anonymization, is a significant step toward expanding the business market.
This initiative can be a business opportunity for companies handling personal information, and I feel a great deal of potential in this software that is necessary for efficiently creating anonymously processed information.

Interview by Hiroaki Tanabe
on October 16, 2019