Data visualization for beginners - Part 1

May 10, 2018
7 mins

This is a series of blogs dedicated to different data visualization techniques used in various domains of machine learning. Data Visualization is a critical step for building a powerful and efficient machine learning model. It helps us to better understand the data, generate better insights for feature engineering, and, finally, make better decisions during modeling and training of the model.

For this blog, we will use the seaborn and matplotlib libraries to generate the visualizations. Matplotlib is a MATLAB-like plotting framework in python, while seaborn is a python visualization library based on matplotlib. It provides a high-level interface for producing statistical graphics. In this blog, we will explore different statistical graphical techniques that can help us in effectively interpreting and understanding the data. Although all the plots using the seaborn library can be built using the matplotlib library, we usually prefer the seaborn library because of its ability to handle DataFrames.

We will start by importing the two libraries. Here is the guide to installing the matplotlib library and seaborn library. (Note that I'll be using matplotlib and seaborn libraries interchangeably depending on the plot.)

Simple Plot

Let's begin by plotting a simple line plot which is used to plot a mathematical. A line plot is used to plot the relationship or dependence of one variable on another. Say, we have two variables 'x' and 'y' with the following values:

To plot the relationship between the two variables, we can simply call the plot function.

Data Visualization Technique: Simple Plot - Relationship between X&Y

Fig. 1. Line Plot between X and Y

Here, we can see that the variables 'x' and 'y' have a sinusoidal relationship. Generally, .plot() function is used to find any mathematical relationship between the variables.

Histogram

A histogram is one of the most frequently used data visualization techniques in machine learning. It represents the distribution of a continuous variable over a given interval or period of time. Histograms plot the data by dividing it into intervals called 'bins'. It is used to inspect the underlying frequency distribution (eg. Normal distribution), outliers, skewness, etc.

Let's assume some data 'x' and analyze its distribution and other related features.

Let's plot a histogram to analyze the distribution of 'x'.

Data Visualization Techniques: Histogram of variable x

Fig 2. Histogram showing the distribution of the variable ‘x’.

The above plot shows a normal distribution, i.e., the variable 'x' is normally distributed. We can also infer that the distribution is somewhat negatively skewed. We usually control the 'bins' parameters to produce a distribution with smooth boundaries. For example, if we set the number of 'bins' too low, say bins=5, then most of the values get accumulated in the same interval, and as a result they produce a distribution which is hard to predict.

Data Visualization Techniques: Histogram with low number of bins

Fig 3. Histogram with low number of bins.

Similarly, if we increase the number of 'bins' to a high value, say bins=1000, each value will act as a separate bin, and as a result the distribution seems to be too random.

Data Visualization Techniques: Histogram with low bins

Fig. 4. Histogram with a large number of bins.

Kernel Density Function

Before we dive into understanding KDE, let's understand what parametric and non-parametric data are.

Parametric Data: When the data is assumed to have been drawn from a particular distribution and some parametric test can be applied to it

Non-Parametric Data: When we have no knowledge about the population and the underlying distribution

Kernel Density Function is the non-parametric way of representing the probability distribution function of a random variable. It is used when the parametric distribution of the data doesn't make much sense, and you want to avoid making assumptions about the data.

The kernel density estimator is the estimated pdf of a random variable. It is defined as
Kernel density equation
Similar to histograms, KDE plots the density of observations on one axis with height along the other axis.

Data visualization using Kernel Density Function

Fig 5. KDE plot for the random variable ‘x’.

Distplot combines the function of the histogram and the KDE plot into one figure.

Data Visualization: Distplot using seaborn

Fig 6. Displot for the random variable ‘x’.

So, the distplot function plots the histogram and the KDE for the sample data in the same figure. You can tune the parameters of the displot to only display the histogram or kde or both. Distplot comes in handy when you want to visualize how close your assumption about the distribution of the data is to the actual distribution.

Scatter Plot

Scatter plots are used to determine the relationship between two variables. They show how much one variable is affected by another. It is the most commonly used data visualization technique and helps in drawing useful insights when comparing two variables. The relationship between two variables is called correlation. If the data points fit a line or curve with a positive slope, then the two variables are said to show positive correlation. If the line or curve has a negative slope, then the variables are said to have a negative correlation.

A perfect positive correlation has a value of 1 and a perfect negative correlation has a value of -1. The closer the value is to 1 or -1, the stronger the relationship between the variables. The closer the value is to 0, the weaker the correlation.

For our example, let's define three variables 'x', 'y', and 'z', where 'x' and 'z' are randomly generated data and 'y' is defined as
EquationWe will use a scatter plot to find the relationship between the variables 'x' and 'y'.

Data Visualization: Scatter plot between X & Y

Fig 7. Scatter plot between X and Y.

From the figure above we can see that the data points are very close to each other and also if we fit a curve, along with the points, it will have a positive slope. Therefore, we can infer that there is a strong positive correlation between the values of the variable 'x' and variable 'y'.

Also, we can see that the curve that best fits the graph is quadratic in nature and this can be confirmed by looking at the definition of the variable 'y'.

Joint Plot

Jointplot is seaborn library specific and can be used to quickly visualize and analyze the relationship between two variables and describe their individual distributions on the same plot.

Let's start with using joint plot for producing the scatter plot.

Data Visualisation: Joint plot using seaborn

Fig 8. Joint plot (scatter plot) between X and Y.

Next, we can use the joint point to find the best line or curve that fits the plot.

Data visualization: Using joint plot for regression

Fig 9. Using joint plot to plot the regression line that best fits the data points.

Apart from this, jointplot can also be used to plot 'kde', 'hex plot', and 'residual plot'.

PairPlot

We can use scatter plot to plot the relationship between two variables. But what if the dataset has more than two variables (which is quite often the case), it can be a tedious task to visualize the relationship between each variable with the other variables.

The seaborn pairplot function does the same thing for us and in just one line of code. It is used to plot multiple pairwise bivariate (two variable) distribution in a dataset. It creates a matrix and plots the relationship for each pair of columns. It also draws a univariate distribution for each variable on the diagonal axes.

Sklearn stores data in the form of a numpy array and not data frames, thereby storing the data in a dataframe.

Data visualization: Pair plot for relation between columns

Fig 10. Pair plot showing the relationships between the columns of the dataset.

So, in the graph above, we can see the relationships between each of the variables with the other and thus infer which variables are most correlated.

Conclusion

Visualizations play an important role in data analysis and exploration. In this blog, we got introduced to different kinds of plots used for data analysis of continuous variables. Next week, we will explore the various data visualization techniques that can be applied to categorical variables or variables with discrete values. Next, I encourage you to download the iris dataset or any other dataset of your choice and apply and explore the techniques learned in this blog.

Have anything to say? Feel free to comment below for any questions, suggestions, and discussions related to this article. Till then, Sayōnara.

  •  
    2
    Shares
  • 2
  •  
  •  
  •  
  •  

About the Author

Shubham Gupta
Trying to solve problems through machine learning and help others evolve in the field of machine learning. Currently working as a Data Science Intern at HackerEarth. Highly enthusiastic about autonomous driven systems.

Want to stay ahead of the technology curve?

Subscribe to our Developers blog


Yes, I would like to receive the latest information on emerging technology trends, as well as relevant marketing communication about hackathons, events and challenges.     By signing up you agree to our Terms of service and Privacy policy.