Getting Started with your first Machine Learning Algorithm - Linear Regression || First step towards ML

-
Linear Regression is a Supervised Learning Algorithm. This is probably the first algorithm taught when you start learning ML. Now as a beginner you must be wondering "What is a Supervised Learning??". So machine learning can be divided mainly into 3 types:
  1. Supervised Learning
  2. Unsupervised Learning
  3. Reinforcement Learning
Supervised Learning is a technique in which we use a labelled dataset to train a model. In short, you can say there is a supervisor who will tell whether the predictions model is making is correct or not. This supervisor decides about based on the labels provided in the datasets. Labels are the correct values for a particular training example. We will deal later with the other two techniques of ML.

In supervised learning, we can classify problems into 2 different categories:
  1. Regression - problems in which we need to predict continuous value as an output.
  2. Classification - where we need to classify the training examples in a particular category.
We plot all the training examples in the datasets and try to find a line that best fits our datasets (as shown in fig). Based on that line we make predictions on unseen examples. So basically linear regression is all about finding the best fit line.


Let's work on some math of Linear Regression Algorithm. First of all, we will understand a simple linear regression in which we will have only 2 values in a dataset one for X and another for Y

Math Behind Linear Regression

Before diving in we will check out some of the notations 

We need to find an equation of a Straight Line that best fits out dataset. Let us assume the equation of a straight line to be - 

y = Wx +b    --- (1)

based on this equation we will construct a hypothesis - 

h(x) = Wx + b    ---(2)

Here, W and b are 2 parameters of a straight line. Now our objective is to find such values of W and b that our hypothesis can give us the most accurate prediction so that we have a minimum error.

Cost Function

To check how good our hypothesis is we will use a Cost Function. Cost Function will help us decide what value our parameters are better for making predictions. The cost function is denoted by J. 

J(w,b)=2m1​∑i=1m​(h(xi
)−yi​)2 

Here,
  •  Xi represents x value for any ith example
  • similarly, the y value is represented by Yi.
  • h(xi) is the estimated value by a hypothesis.
  • m is the number of training example in our dataset.
We are trying to figure out that the value which we estimated with the hypothesis is close to the actual value or not. The difference gives us error for a particular example. Then we will take an average of it.

So, this was to find out whether our parameters have optimal values or not. Now, we will try to improve our parameters values.

Gradient Descent

Gradient Descent is an algorithm we use to get better values of our parameters. The process of building a successful model and giving predictions is an iterative task. We need to do it repeatedly to find which values of our parameter can give better results.
It is slightly difficult to discuss the whole process behind Gradient descent. I will give a better intuition behind it in another blog. So, here we will just look at the formulas.

W=W−α∂W∂J(W,b)​

b=b−α∂b∂J(W,b)​


The α is called a learning rate. We will discuss more about it later.
We need to repeatedly follow these two equations to find a better and better value. The cost function can help us decide which is the best value of our parameter. These two formulas are used to update the values of our parameters. The generalised form of this equation is as follow - 

θj​=θj​−α∂θj​∂J​

Currently, you must be wondering how this formula came? and why it will give us better values of parameters? and many more. But, once you will read my blog on Gradient Descent you will get a clear picture of this. I will share the link here as soon as I publish that blog. Till then just go with the flow.

So, now we have our parameters which can give best-fit line to our dataset. Now, our model can give the most accurate. To measure the model's accuracy we can use Mean Squared Error (MSE).
Now, look at the GIF above. It might give you a good understanding of Linear Regression.
In the next blog, I will try to cover Gradient Descent's better picture and Implementation of Linear Regression in Python. Stay tuned for it. If you have any doubts you can leave comments here and your feedbacks are always appreciated. Thanks for reading it.

Comments

Post a Comment

If you have any questions do let me know and also give your valuable feedback for this blog.

Popular posts from this blog

Hierarchical Clustering - An Unsupervised Learning Algorithm

-
Image
Introduction Unsupervised learning  is a type of Machine learning in which we use unlabeled data and we try to find a pattern among the data. Clustering algorithms falls under the category of unsupervised learning. In these algorithms, we try to make different clusters among the data. Hierarchical Clustering algorithms build a hierarchy of clusters where each node is a cluster consisting of the clusters of its children node. To check it's implementation in Python  CLICK HERE There are various strategies in Hierarchical Clustering such as : Divisive Agglomerative This type of diagram is called  Dendrogram. Divisive - It is a Top-down approach. So we start with all observations in a large cluster and break it down into smaller ones. Agglomerative - It is the opposite of Divisive as it is a Bottom-Up approach. Here, each observation starts in its cluster and pairs of the cluster are merged as they move up the hierarchy.  (Generally, Agglomerative is used more as compa...
Read more

Implementing Hierarchical Clustering - In Python Programming language

-
Image
Introduction to Hierarchical Clustering Unsupervised learning  is a type of Machine learning in which we use unlabeled data and we try to find a pattern among the data. Clustering algorithms  falls under the category of unsupervised learning. In these algorithms, we try to make different clusters among the data. Hierarchical Clustering  algorithms build a hierarchy of clusters where each node is a cluster consisting of the clusters of its children node. fig. 1 Check out my blog on Hierarchical Clustering - An Unsupervised learning Algorithm  to learn more about it. Implementing it with Python and SciKit-Learn We will use the python programming language for its implementation. In python language, we will be using SciKit-Learn library. So let's start with implementation - We will be using our own dataset which will be generated by us.     1.  Importing necessary Libraries - To create a machine learning model we will use SciKit-Learn (sklearn) librar...
Read more

Pose Estimation || Application of Computer Vision and Deep Learning

-
Image
What is Pose Estimation? Pose Estimation is one of the recent applications of Deep Learning and Computer Vision. In pose estimation we try to predict the pose of a person, it means representing an orientation of a person in graphical format. While reading out the intro it might look a bit complex but actually, it is not. We implement the idea by considering each joint of our body as a coordinate for our graphical representation. The bones which connect two joints are shown as a simple line. To make a posture simple and easy to visualize we do not represent each and every bone of a human body. Instead, we represent Hands, Legs, Shoulder and Face of a person. Background Previously, one of the famous application of computer vision was face detection and face recognition. We also used to apply computer vision for detecting whether a person is present in an image or not. If the presence is detected then we can highlight the area where it is present. So, after making some advancements and a ...
Read more