There is considerable overlap between the fields of neural networks and statistics. Statistics is concerned with data analysis. In neural network terminology, statistical inference means learning to generalize from noisy data. Some neural networks are not concerned with data analysis (e.g., those intended to model biological systems) and therefore have little to do with statistics. Some neural networks do not learn (e.g., Hopfield nets) and therefore have little to do with statistics.
Some neural networks can learn successfully only from noise-free data (e.g., ART or the perceptron rule) and therefore would not be considered statistical methods. But most neural networks that can learn to generalize effectively from noisy data are similar or identical to statistical methods. For example:
* Feedforward nets with no hidden layer (including functional-link neural nets and higher-order neural nets) are basically generalized linear models.
* Feedforward nets with one hidden layer are closely related to projection pursuit regression.
* Probabilistic neural nets are identical to kernel discriminant analysis.
* Kohonen nets for adaptive vector quantization are very similar to k-means cluster analysis.
* Kohonen self-organizing maps are discrete approximations to principal curves and surfaces.
* Hebbian learning is closely related to principal component analysis.
