Data Science Updates

What is a Time Series?

• Time Series data is a series of data points indexed or listed or graphed with an equally spaced period.
• Time series forecasting is the use of the model to predict future values based on previously observed value.
• Input Past --> Predict future.

Component of Time Series:

• Trend: Increase or decrease over a long period. India GDP
• Seasonal Pattern: Short-term variation occurring due to seasonal factors. Winter sales
• Cyclical Patterns: It is a medium-term variation caused by circumstances, which repeat in irregular intervals. Recession
• Irregularity Patterns: Which Occurs due to unpredictable factors. Anomaly

Where can we not use Time Series?

• If the variable is not dependent on time.
• If Values are constant.
• Values represented using known function as sin(x), cos(x) etc.
• Time Series algorithm cannot be used if data is not stationary.

Time series models:

• ARIMA
• SARIMA
• FARIMA
• LSTM

Outliers
* An Outlier is an observation point that is distant from other observations.
* An outlier may indicate an experimental error, or it may be due to variability in the measurement.
* Outliers are different from the noise data. Noise is random error or variance that needs to be removed before outlier detection
* We can check the performance of dataset without outlier by checking scores.

Categories of Outlier

a) Global Outlier / Point Anomalies
* They are defined as data points that differ from the rest of the data.
* A particular type of global outlier is the influential point. It is defined as the outlier that impacts the rest of the data.
* We can evaluate an outlier based on the R2 score and regression line.
* Removing an outlier may or may not increase the R2. We have to study the individual purpose if needed.

b) Contextual / Conditional Outlier
* It is defined as the data point whose context differs from the rest of the data.
* It can be a point that is following a trend in some other context, with what defined for rest of the dataset.
* We have to study it and remove it carefully.
* We shall focus on two conditions :
    Contextual attributes: e.g., time, location, etc.
    Behavioural attributes: e.g., temperature, calories taken

c) Collective Outlier
* It is defined as a set of data point that deviates significantly from the rest of the data, even if the individual data points are not the outliers.
* We have to study it carefully. Removal of such data points may degrade the system. We shall focus on improving it.

Mathematics is the building block for data science. This blog focuses on various mathematical concepts that are used in machine learning. The mathematical concepts used for machine learning are categorized into statistics, probability, differential calculus. Let’s discuss one by one.

1.Statistics

In mathematical terms, statistics is defined as the set of equations, which are helpful to interpret and analyze things.  In machine learning, statistics plays a very important role in understanding the data in a dataset. Various statistical analysis helps us to understand the distribution, summary, etc. of data.

1.1.Exploratory data analysis

EDA or exploratory data analysis is one of the critical steps in data science. It helps us to analyze the data patterns, errors, outliers, etc. Statistics being the backbone for this step, various concepts such as standard deviation, variance, mean, median, etc. are used.

We consider data that is outside three standard deviations (In general) as the outliers. We understand data distribution by plotting a bar graph, which helps us understand whether data is distributed across mean or is the data skewed towards one side.

2.Probability

Probability is the branch of mathematics which is concerned with the numerical description of explaining how likely an event is to occur. This theory is very useful in making predictions. Estimation and predictions constitute an important part of Data Science, and thus, most of the concepts involve probability theory.

2.1.Classification algorithms

Most of the classification problems in data science involve the predictions of classes, where we classify each observation to exactly one class. The base idea behind the classification problem is probability. The probabilities of all the classes are calculated based on the trained data; the class with the highest probability is assigned to that observation.

2.2.Loss function

One of the loss functions used for classification problems is the cross-entropy loss which is a measure of the classification model. Cross-entropy loss increases as the predicted probability diverge from the actual label. It is one of the most important calculations when it comes to machine learning for classification.

3.Differential calculus

Data science is incomplete without differential calculus. Differentiation forms an intrinsic part of data science, especially in machine learning. Differentiation or calculus is the study of the rate of changes in quantities.

3.1.Gradient Descent

In machine learning, our goal is to reduce the cost to our input data. We use cost function, which is the measure of the error in the predictions of the model. To achieve the lowest possible value of the cost function is the main goal of gradient descent which in turn improves the accuracy. Gradient descent uses differentiation where the partial derivative of the cost function is calculated, which will point to the global minima. The downfall of the gradient is controlled by the learning rate.

The same concept is applied for deep learning models where the optimizer used as gradient descent will use the partial derivative concept to adjust the weights to get the optimal weights.

Regularization

In Machine Learning, Regularization is the concept of shrinking or regularizing the coefficients towards zero.

It helps the model to prevent overfitting.

Overfitting in Machine Learning is referred to as an algorithm while getting model has a lot of features or a low number of observations. 

A linear model tends to overfit and when an algorithm gets modelled has a lot of features or a low number of observations, at that time the variable selection becomes tricky.

In Machine Learning, Regularization is achieved by the help of:

1. Lasso Regression (L1)

2. Ridge Regression (L2)

3. Elastic Net Regression (L1+L2).

 Linear Regression

A Linear regression is  a Regression Analysis technique which is used for modeling the predictions on the continuous data.

A Linear Regression can be modelled using 

1. A Simple Regression technique

2. A multi regression technique

1. Simple Regression:  It is a kind of regression technique where we have a single independent variable(X) and a single dependent variable(Y).

The main aim of this kind of modelling is to develop a regression line of the following form

Y=mX+c

Y -> Dependent Variable

m -> slope

X -> Independent Variable

c -> Y-intercept

2. Multi Regression:  It is a kind of regression technique where we have multiple independent variable(X1,x2,x3...) and a single dependent variable(Y).

The main aim of this kind of modelling is to develop a regression line of the following form

Y=c+m1X1+m2X2+...

Y -> Dependent Variable

m -> slope

X1,X2,X3...-> Independent Variables

c -> Y-intercept

Data Science and Machine Learning are one of the most sought after technology to learn these days. With the changing customers and rapidly evolving market, the companies are looking at the customer’s behaviour and are making a Data-driven decision, which makes Data Science a trending technology. Whether you are a fresh graduate or a working professional, if you have an interest in numbers, and are willing to learn new skills, this field is for you.

There are two ways to acquire knowledge in this area. 

1. By getting a degree
2. By being self-taught

Getting a degree:

Getting a formal education is always considered as a safe option. You’ll get a favourable environment for studying and have the opportunity to study with the like-minded people. You’ll also study in a more structured way and get a good exposure. Thus, formal education is a popular way of studying Machine Learning. Though Machine learning courses are not pocket friendly and you may also find it challenging to get a good institute which offers the course.

Being Self-taught:

This way of learning is trending nowadays. With the help of the internet and courses available online, a large number of people are opting for this route. If you are a working professional, this is the best-suited option for you. Online learning courses are pocket-friendly and flexible in terms of timing. However, you need to maintain self-discipline and manage time efficiently if you choose this option.

Overall, Machine learning is already becoming accessible to self-taught individuals. If you are willing to put the time and effort it, then you should go for it.

In recent years, Data Science has emerged as one of the most sought-after career options for people who are willing to learn new technologies and are passionate about data and numbers. As the businesses have started monitoring their customer’s behaviour and are making data-driven decisions, the demand for Data Science professionals is increasing. It will continue to grow in the upcoming years.

When it comes to qualification, there are no specified criteria for this, but it is always better to have Maths and Science background. If you earn a bachelor’s degree in IT, Computer Science, Maths, or other related subjects, it would be better. Then, you can opt for a master’s degree in Data or related field.

If we look at the various employment reports, we will find that there is a shortage of 1 million data scientists by 2018-2020. So, this is a huge opportunity for freshers to kick-start their career in data science. However, as they say, “What is easy, is not worth taking.” Alongside practical and theoretical knowledge, you need specific skills to become a successful data science professional. So, you should upskill yourself well enough to ensure that you are ready for the opportunity that comes on your way.

Hope this helps. All the best!

It's true that Data Science is one of the most trending fields in today’s era. With the market becoming digital and companies growing their businesses around the world, Data Science has become the need of the hour. A person who studies and gains proficiency in this field has a very bright future. But, still, there are lots of confusions amongst the aspiring students and professionals about the right path they should take to ensure career growth.

Here is a roadmap that could help you in your journey to becoming a data scientist. It may not be the easy path to choose, but the results would be worth it.

- Study Maths and Statistics

It is the first and foremost skill to become a successful Data Scientist. People often sideline this primary step. To understand the data in a practical way, one must appreciate Maths and Stats well. You should start with topics such as linear algebra and gradually move to calculus.

- Get your hands-on Programming

There are specific programming languages, like R and Python, that are a must-know for a Data professional. Most of the professionals will agree on this fact that these languages are often used in this field.

- Learn Machine Learning:

It is also an essential part of Data Science. So, one should focus on not only, learning ML but also the ways to apply it for solving problems.

- Work on Some real-time projects:

Once you learn, you should know the ways to implement it. So, working on some real-time projects will help you understand the concepts better.

- Build a strong portfolio and start applying for jobs:

Now that you have some strong skills to put on the table, you should build a strong portfolio, including these skills, and start hunting for some excellent opportunities.

It’s not surprising that you asked this question as the road to becoming a data scientist isn't easy. The primary reason behind it is that this field still lacks a proper development base. Also, there are a few points, which makes it a challenging skill to acquire:

Problems:

The problems tackled by a data scientist are tough to solve. One needs to be well-versed in mathematical studies, like algebra, calculus, statistics to fix the issues arose.

Large Set of Data:

In today’s world, there is a large amount of data present, and it becomes tedious at times for a Data Scientist to handle them. Above that, the data is unstructured most of the times.

Acquiring multiple skills:

A data scientist has to learn multiple skills at a time to be able to perform the tasks assigned. They should have proper knowledge of Maths, various programming languages like R, Python. Also, they should have adequate communication skills.

Towards the end, it takes a lot of skills and knowledge to become a data scientist, but there is a saying “What is easy, is not worth taking.” So, if you think you have got what it takes to become a data scientist and if you prepare yourself well enough, then the ultimate result will be rewarding.

All the best!

With the ever-developing innovation, Data Science is a new term which is making a lot of buzzes lately. Data science, as an interdisciplinary field, is suitable for individuals who enjoy playing with numbers, learning new advancements and information.

Since data science has taken the world by storm, companies across the globe are hiring professionals in this domain, be it data analysts, data scientists, BI developers etc. Moreover, as organizations worldwide have started checking their client's conduct and making data-driven decisions, the demand for data scientists is booming and the graph will only grow in the upcoming years.

If we look at the various employment reports, we will find that there is a shortage of 1 million data scientists by 2018-2020. In simple terms, Data scientists are highly in-demand in today's market so, this is a massive opportunity for the freshers as well as experienced graduates, who are looking for a better chance. Some of the companies that hire Data Science professionals are:

• Busigence Technologies
• Happay
• Sigmoid
• Walmart Labs
• Quantiphi Inc
• Mate Labs
• Tracxn
• Couture
• Crediwatch

If you're a data scientist fresher, then you must acknowledge the fact that a lot of companies hire experienced data scientist professionals and not freshers. It is probably because of the nature of this job. But, don't worry! There are companies that look for fresh minds/talents so, all you need to do is to upskill yourself, stay-up-to-date and you'll get your dream job.

Hope this helps!