A neural network is an interconnected network of neurons with each neuron being a limited function approximator. This way, neural networks are considered as universal function approximators.