In computer science, a convolutional neural network is a type of feed-forward neural network where the individual neurons are tiled in such a way that they respond to overlapping regions in the visual field.^[1] Convolutional networks were inspired by biological processes^[2] and are variations of multilayer perceptrons which are designed to use minimal amounts of preprocessing.^[3] They are widely used models for image-recognition.^[4]

Architecture[edit]

Convolutional neural networks consist of multiple layers of small neuron collections which look at small portions of the input image. The results of these collections are then tiled so that they overlap to obtain a better representation of the original image; this is repeated for every such layer. Because of this, they are able to tolerate translation of the input image.^[4] Most convolutional networks include local pooling or max pooling layers, which simplify and combine the outputs of neighboring neurons; essentially, if the outputs are integrated into an image, pooling layers reduce its resolution.^[5] They also consist of various combinations of convolutional layers and fully connected layers, with pointwise nonlinearity applied at the end of or after each layer.^[6]

Some time-delay neural networks also use a very similar architecture to convolutional neural networks, especially those for image recognition and/or classification tasks, since the "tiling" of the neuron outputs can easily be carried out in timed stages in a manner useful for analysis of images.^[7]

History[edit]

Convolutional neural networks were introduced in a 1980 paper by Kunihiko Fukushima.^[6][8] Their design was later improved in 1998 by Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner,^[9] generalized in 2003 by Sven Behnke,^[10] and simplified by Patrice Simard, David Steinkraus, and John C. Platt in the same year.^[11] In 2011, they were refined by Dan Ciresan et al. and were implemented on a GPU with impressive performance results.^[12] In 2012, Dan Ciresan et al. broke all existing records for multiple image databases, including the MNIST database, the NORB database, the HWDB1.0 dataset (Chinese characters), and the CIFAR10 dataset (a subset of the 80 million tiny objects database).^[6]

Usage[edit]

Convolutional neural networks are often used in image recognition systems. When applied to hand tracking in a video stream and gesture recognition, they had almost perfect performance.^[13] They have achieved performance double that of humans on the problem of recognizing traffic signs and an error rate of 0.23 percent on the MNIST database, which is the lowest ever achieved on the database to date.^[6] Another paper on using convolutional neural networks for image classification reported that the learning process of convolutional neural networks was "surprisingly fast"; in the same paper, the best published results at the time were achieved in the MNIST database and the NORB database.^[12]

They have also been confirmed to have lower error rates than both deep neural networks and regular neural networks for large-vocabulary voice recognition tasks.^[14]

When applied to facial recognition, they were able to contribute to a large decrease in error rate.^[15] In another paper, they were able to achieve a 97.6 percent recognition rate on "5,600 still images of more than 10 subjects".^[2] Convolutional neural networks have been used to assess video quality in an objective way after being manually trained; the resulting system had a very low root mean square error.^[7]

See also[edit]

• Neocognitron
• Convolution

References[edit]

External links[edit]

• http://yann.lecun.com/exdb/lenet/ - A demonstration of a convolutional network created for character recognition.