Deep Learning has received a lot of attention in industry because of successes in fields of speech recognition, computer vision and AI. However, Deep Learning is time-consuming for the training. In order to overcome this problem, we have developed efficient training algorithms for Deep Learning. Our algorithm adapts learning rate by using directions of past gradients. This approach gives 2 to 5 times fewer training iterations than previous algorithm. In addition, our method reduces tuning costs of hyper parameters and achieves high accuracy.
Please click the thumbnail image to open the full-size PDF file.