Natural language processing

Technology

For more than 40 years, NTT has been at the forefront of research and development of technologies that enable computers to understand and generate natural language used by humans. The applications of these technologies are diverse, including automatic dialogue, question answering, document summarization, translation, and information retrieval. In recent years, NTT has focused on methods that use large-scale language models, which are neural networks learned from large text corpora. We are also challenging the technology of "Vision-and-Language," a fusion of vision and language understanding. Our research and development are about understanding human communication in a multimodal way and generating it, with language at the core.

NTT Human Informatics Laboratories aim to realize a society that uses AI to understand the world and grow with others.

Research

  1. Understanding and generating natural language (under construction)
  2. Understanding and generating human communication in a multimodal (under construction)

Publications

2023

2022

2021

2020

2019

2023

Conference Papers

  1. Kosuke Nishida, Naoki Yoshinaga (Tokyo Univ.) and Kyosuke Nishida, "Self-Adaptive Named Entity Recognition by Retrieving Unstructured Knowledge", in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023), accepted, May 2023. [arxiv]
  2. Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, Kuniko Saito, "SlideVQA: A Dataset for Document Visual Question Answering on Multiple Images", in Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI 2023, 1721/8777=19.6%), accepted (full paper), February 2023. [arxiv]

2022

Conference Papers

  1. Yasuhito Ohsugi, Itsumi Saito, Kyosuke Nishida and Sen Yoshida, "Japanese ASR-Robust Pre-trained Language Model with Pseudo-Error Sentences Generated by Grapheme-Phoneme Conversion", in Proceedings of the 2022 Conference of the International Speech Communication Association (INTERSPEECH 2022), pp. 2688-2692, September 2022.
  2. Fumio Nihei, Ryo Ishii, Yukiko Nakano, Kyosuke Nishida, Ryo Masumura, Atsushi Fukayama and Takao Nakamura, "Dialogue Acts Aided Important Utterance Detection Based on multiparty and multimodal information", in Proceedings of the 2022 Conference of the International Speech Communication Association (INTERSPEECH 2022), pp. 1086-1090, September 2022.
  3. Kosuke Nishida, Kyosuke Nishida, Shuichi Nishioka, "Improving Few-Shot Image Classification Using Machine- and User-Generated Natural Language Descriptions", in Proceedings of the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2022) (findings), pp. 1421-1430, July 2022. [arxiv]
  4. Shumpei Miyawaki (Tohoku Univ.), Taku Hasegawa, Kyosuke Nishida, Takuma Kato (Tohoku Univ.), Jun Suzuki (Tohoku Univ.), "Scene-Text Aware Image and Text Retrieval with Dual-Encoder", in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW 2022), pp. 422-433, May 2022.

2021

Conference Papers

  1. Kosuke Nishida, Kyosuke Nishida, Sen Yoshida, "Task-adaptive Pre-training of Language Models with Word Embedding Regularization", in Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021) (findings), pp. 4546-4553, August 2021. [arxiv]
  2. Kosuke Nishida, Kyosuke Nishida, Itsumi Saito, Sen Yoshida: "Towards Interpretable and Reliable Reading Comprehension: A Pipeline Model with Unanswerability Prediction." in Proceddings of the 2021 International Joint Conference on Neural Networks ( IJCNN 2021), pp. 1-8, July 2021. [arxiv]
  3. Ryota Tanaka(*), Kyosuke Nishida(*), Sen Yoshida, "VisualMRC: Machine Reading Comprehension on Document Images", in Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI 2021), pp. 13878-13888, a Virtual Conference, February 2021. (*: equal contribution) (full paper, 1696/7911=21.4%) [arxiv] [project]

2020

Conference Papers

  1. Diana Galvan-Sosa (Tohoku Univ.), Jun Suzuki (Tohoku Univ.), Kyosuke Nishida, Koji Matsuda (Tohoku Univ.) and Kentaro Inui (Tohoku Univ.): Seeing the world through text: Evaluating image descriptions for commonsense reasoning in machine reading comprehension, in Proceedings of the Second Workshop on Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN 2020; in conjunction with COLING 2020), December 2020.
  2. Yuma Koizumi, Ryo Masumura, Kyosuke Nishida, Masahiro Yasuda and Shoichiro Saito, "A Transformer-based Audio Captioning Model with Keyword Estimation", in Proceedings of the 21st Annual Conference of the International Speech Communication Association (INTERSPEECH 2020), October 2020. [arxiv]
  3. Kosuke Nishida, Kyosuke Nishida, Itsumi Saito, Hisako Asano and Junji Tomita, "Unsupervised Domain Adaptation of Language Models for Reading Comprehension", in Proceedings of the 12th International Conference on Language Resources and Evaluation (LREC 2020), pp. 5392-5399, May 2020. [arxiv]

2019

Conference Papers

  1. Ryo Masumura, Mana Ihori, Tomohiro Tanaka, Itsumi Saito, Kyosuke Nishida, and Takanobu Oba, "Generalized Large-Context Language Models based on Forward-Backward Hierarchical Encoder-Decoder Models", in Proceedings of the 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU 2019), pp.554-561, December 2019.
  2. Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda (Tokyo Univ.), Atsushi Otsuka, Hisako Asano and Junji Tomita, "Multi-style Generative Reading Comprehension", in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp.2273-2284, July 2019. [arXiv] (long paper, 660/2905=22.7%)
  3. Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Itsumi Saito, Atushi Otuka, Hisako Asano and Junji Tomita, "Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction", in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pp.2273-2284, 2335-2345, July 2019. [arXiv] (long paper, 660/2905=22.7%)
  4. Yasuhito Ohsugi, Itsumi Saito, Kyosuke Nishida, Hisako Asano, and Junji Tomita, "A Simple but Effective Method to Incorporate Multi-turn Context with BERT for Conversational Machine Comprehension", in Proceedings of 1st Workshop on NLP for Conversational AI (NLP4ConvAI 2019; in conjunction with ACL 2019), Florence, Italy, July 2019. [arXiv]
  5. Atsushi Otsuka, Kyosuke Nishida, Itsumi Saito, Hisako Asano and Junji Tomita, "Specific Question Generation for Reading Comprehension", in Proceedings of the AAAI 2019 Reasoning for Complex QA (RCQA) Workshop, Honolulu, Hawaii, USA, January 2019.