Before you walk through this post, you should have some knowledge of gradient descent and Perceptron. It's highly recommended to check out these two posts Gradient Descent Example Code and Single-layer Neural Networks (Perceptrons) before go through the following content. As a beginner of Machine Learning, I recommend you to start with some tiny examples rather than some abstract concepts. Since you have learned some basics of Neural Network, read some influential papers of this filed. You will be amazed by the charm of Neural Network.

In this post, we are going to talk about how to use perceptron to implement conjunction operation, the relationship between perceptron and neural network, the way neural network works, and most importantly how to implement a single layer neural network optimized by BP algorithm. We will as well talk about the problem of vanishing gradient at the end and provide some solutions. You are expected to have a basic idea of what is neural network and how neural network works and optimized.

PDF version at http://www.iovi.com/post/2016-04-04-simple-neural-network.pdf