Introduction to Deep Learning

Deep learning is an incredibly powerful tool of Machine Learning that is based on learning data representations instead of task-specific algorithms. Deep Learning uses networks where data transforms through a number of layers before producing the output.


Traditional machine learning algorithms typically try to define as the set of rules or features in the data and these are usually hand-engineered that’s why it tends to be brittle in practice. For example, if you want to perform facial detection, the first thing you have to do is classify or recognize mouth, eyes, ears and all in the image, if you find everything then you can say there is a face in the image. To recognize each thing you have to define a set of features.

The key idea of deep learning is that you will need to learn the features just from raw data, you just have to take a bunch of images of faces and then the deep learning algorithm is going to develop some hierarchical representation of first detecting lines and edges, using these lines and edges to detect corners and mid-level features like eyes, noses, mouths, etc then composing these together to detect higher-level features like jawlines of the face, etc which can be used to detect the final face structure.

The fundamental building block of a neural network which is a single neuron also called Perceptron


Forward Propagation:
We define a set of inputs to that neuron as x1 through xm. Each of these inputs has a corresponding weight w1  to wm. With each of these inputs and weights, we can multiply them correspondingly together and take a sum of all then we take a single number i.e. summation and we pass it through non-linear activation function that produces our final output ‘Y’. But this is not entirely correct, we also have bias term in this neuron. The purpose of the bias term is to allow to shift activation function to the left or right regardless of your inputs.

We can rewrite the output equation as a linear algebra in terms of vectors and dot products. Now we have a vector of inputs and weights and then we apply non-linearity which is ‘g’(non-linearity activation function:

One common example of the activation function is called the sigmoid function. This function takes any real number as input and it transforms a real number into a scalar output between 0 and 1. And there are many activation functions other than sigmoid.



Comments

Post a Comment

Popular posts from this blog

Importance of Activation Functions

Image
The purpose of activation functions is to introduce non-linearities into the network. Non-linear function allows to approximate arbitrarily complex functions that make neural network extremely powerful. For example, when a trained network with weights W and the network has only two inputs- x 1  ,  x 2  and we pass it through a non-linearity. Before applying non-linearity, If we feed in with a new input x 1 = -1 and x 2 = 2, the idea can be generalized a little bit more if we compute the line, we get minus -6. When we apply a sigmoid non-linearity, it collapses between 0 and 1. Sigmoid function results anything greater than 1 as above 0.5 and anything less than 1 as below 0.5. The reason why we use non-linear activation function is when deal with network thousands or millions of parameters and dimensional spaces then visualizing these type of plots becomes extremely difficult.
Read more

The idea of Neural Network

Image
The basic idea behind a neural network is to simulate lots of densely interconnected brain cells inside a computer so you can get it to learn things, recognize patterns, and make decisions in a human-like way.   The amazing thing about a neural network is that you don't have to program it to learn explicitly: it learns all by itself, just like a brain! It's important to note that neural networks are (generally) software simulations: they're made by programming very ordinary computers, working in a very traditional fashion with their ordinary transistors and serially connected logic gates, to behave as though they're built from billions of highly interconnected brain cells working in parallel. No-one has yet attempted to build a computer by wiring up transistors in a densely parallel structure exactly like the human brain. Building Dense Layer from scratch: Single Layer Neural Network: A single hidden layer is fed into a single output layer. The stat...
Read more