Model of a hierarchical neural network in action
Credit: Kurt W. Fleischer/Caltech Graphics
SYDNEY, 1 August 2006 - Analysing large sets of data to pick out the most important patterns can be problematic for computers and a researcher's nightmare, but according to a new study in the U.S. journal Science, this can be efficiently achieved using artificial neural networks.
Geoffrey Hinton and Ruslan Salakhutdinov, computer engineers from the University of Toronto, Canada, explained that artificial neural networks, so called because they were inspired by the operation of the brain, are devices that are connected by a large number of switches. They differ from conventional computers which follow a predetermined set of rules to solve a problem in that they are capable of 'learning' from examples and can be used to recognise and forecast patterns.
The authors show how they used neural networks to convert high-dimensional data (for example, digitised images of faces taken with a 3-megapixel camera) into lower-dimensional representations.
"This is exciting work with many potential applications in domains of current interest such as biology, neuroscience, and the study of the Web," said Garrison Cottrell, a computer engineer who authored a review of Hinton and Salakhutdinov's study, also released last Friday in the same journal.
Hinton and Salakhutdinov were able to analyse data sets with many dimensions to find lower dimensional structures within them by compressing data. The authors also show that once they obtained the information from this compressed form, they were able to reproduce the original data more closely than previous techniques ever have.
"Hinton and Salakhutdinov's approach uses so-called autoencoder networks - neural networks that learn a compact description of data," said Cottrell. "They do this by making good use of recently developed machine learning algorithms for a special class of neural networks."
This is said to be the first practical method that has resulted in a completely invertible mapping system, so that new reproduced data can be transferred into this low dimensional space. It is the researchers hope that these low dimension representations will be used for essential tasks such as pattern recognition, transformation or visualisation.
Neural networks used to receive quite a lot of attention but have died down in recent times. "Neural networks have been considered 'dead' in some arenas of machine learning," said Cottrell, who claims that while this work is still in its early stages it has a lot of potential for numerous applications.
