Data is the main spice in the recipe of machine learning and deep learning. It is said that a good amount of data is responsible for developing a better-performing AI model.

Nowadays, many AI-based services and applications are driven by deep learning. These services and applications are based on deep-learning models and the accuracy of the deep-learning models highly depends on the quantity and quality of the training data. However, data scarcity is a significant challenge in building accurate deep-learning models.

Companies fulfill the need for data by using the data augmentation technique that reduces the work of collecting and preparing training data beforehand supplying it to the model.

In this article, you’ll be introduced to –

What is data augmentation?
Why do we need data augmentation?
How does data augmentation work?
Advanced models for data augmentation
Demonstration of data augmentation

What is data augmentation?

Data augmentation is a process of increasing the quantity of data artificially by generating new data points from the existing data. This includes adding minor alterations to the data or using machine learning models to generate new data points.

If you are working with image data, then augmenting that data would be flipping the images horizontally and vertically, increasing or decreasing the brightness, skewing, rotating, shifting the position, and more.

Why do we need data augmentation?

In production use cases, collecting the training data can be costly and highly time-consuming. However, the company uses data augmentation, an effective and low-cost method that cuts off the work of collection and preparation of new data.

Data augmentation is useful for improving the performance of the machine learning model which will eventually affect the outcome in a good way by diversifying the dataset.

The machine learning model’s accuracy completely depends on the dataset’s quality, quantity and representability. Collecting and cleaning the new data may increase the risk of the existing data being less accurate and representable. Data augmentation techniques help produce new variations from the current data that make the machine-learning model more robust and perform well for real-world inputs.

How does data augmentation work?

Data augmentation is about making minor changes to the existing data to produce new variations. These are done by passing the dataset to transformation functions that perform transformation and then produce a new dataset.

Here’s an illustration that will explain the workflow of the data augmentation process in which a sequence of transformation functions are applied by human experts.

Figure: Workflow of the heuristic data augmentation

A usual workflow of heuristic data augmentation has the following steps:

a. The data is fed to the data augmentation pipeline containing the sequence of transformation functions.

b. The data augmentation pipeline is defined by the sequential steps of different augmentations to produce different variations of each data point.

TF1 – Flipping
TF2 – Blur
TF3 – Rotation
TF4 – Skewing
TF5 – Grayscale to RGB
TFn – Brightness

c. The image is then fed to the data augmentation pipeline and processed through each step of the transformation function.

d. After the image is processed, a human expert verifies the augmented results.

e. After the verification process, the augmented results are now ready to use by the AI model for training.

Heuristic data augmentation is used to work with models that will classify images.

Data augmentation is less prevalent in the NLP (Natural Language Processing) field than in image data. The main reason is the complexity of the natural language makes it difficult to automate the process of augmenting the text data.

Models for data augmentation

Some popular open-source Python packages for data augmentation available are ImageDataGenerator from Keras, Skimage, and OpenCV.

But there are also some advanced models for data augmentation available that gained popularity lately.

Adversarial machine learning: It injects deceptive inputs to deceive the classifiers in the machine-learning models.
Generative adversarial networks (GANs): GANs algorithms are used to learn the pattern or regularities from the input data and automatically create new data containing features drawn from the original data.
Neural style transfer: is used to generate images with the same content as the original image but in our chosen image’s visual style or appearance.

For Natural Language Processing

Easy Data Augmentation (EDA): consists of four simple but powerful operations that are synonym replacement, random insertion, random swap, and random deletion.
Back translation: data is re-translated back to the original language from the target language means translation from Hindi to English.
Contextualized word embeddings

Demonstration of data augmentation

For demonstration purposes, you’re going to see the data augmentation techniques for the image using a deep-learning library called Keras that provides a utility called ImageDataGenerator.

Let’s start coding the process of augmenting the image data and here is the preview of the sample image that will be used.

Image Credit: Photo by Juan Manuel Sanchez on Unsplash.

The augmentation techniques you’re going to see are:

Random Shift (Width and Height)
Random Flip (Horizontal and Vertical)
Image Rotation
Random Brightening

Visit my GitHub repository to learn additional data-augmenting techniques.

Random Shift

In the following notebook, we are randomly shifting the image by width and height.

import numpy as np

import matplotlib.pyplot as plt

from keras.preprocessing.image import ImageDataGenerator

from keras.utils import load_img, img_to_array

# Preview of the image we will work on

img = load_img('/content/drive/MyDrive/Sample/resized_cat.jpg')

# Random Shift

pic = img_to_array(img)

pic_sample = np.expand_dims(pic, axis=0)

data = ImageDataGenerator(width_shift_range=[10, 50], height_shift_range=[30, 70])

aug_data = data.flow(pic_sample, batch_size=1)

fig, axes = plt.subplots(1, 10, figsize=(30, 40))

for i in range(10):

# Accessing each sample

aug_img = next(aug_data)

# Converting to unsigned integer

aug_result = aug_img[0].astype('uint8')

# Showing augmented images

axes[i].imshow(aug_result)

axes[i].axis('off')

First, we imported the required libraries and loaded our image and then we converted the image into an array and expanded the dimension of the array.

Next, we instantiated the ImageDataGenerator class and specified the operation width_shift_range=[10, 50], height_shift_range=[30, 70] and then passed the sample image to the ImageDataGenerator for generating the processed images.

To see the augmented results, we iterated over the original image, converted it into an unsigned integer, and then plotted the augmented samples.

Random Flip

We are doing the same process but this time we changed the augmenting technique and specified the operation horizontal_flip=True, vertical_flip=True that will randomly flip the images horizontally and vertically.

import numpy as np

import matplotlib.pyplot as plt

from keras.preprocessing.image import ImageDataGenerator

from keras.utils import load_img, img_to_array

# Preview of the image we will work on

img = load_img('/content/drive/MyDrive/Sample/resized_cat.jpg')

# Random flip

pic = img_to_array(img)

pic_sample = np.expand_dims(pic, axis=0)

data = ImageDataGenerator(horizontal_flip=True, vertical_flip=True)

aug_data = data.flow(pic_sample, batch_size=1)

fig, axes = plt.subplots(1, 10, figsize=(30, 40))

for i in range(10):

# Accessing each sample

aug_img = next(aug_data)

# Converting to unsigned integer

aug_result = aug_img[0].astype('uint8')

# Showing augmented images

axes[i].imshow(aug_result)

axes[i].axis('off')

Rotation

To generate the rotated images, we specified rotation_range=50 that will randomly generate the images within the specified range.

import numpy as np

import matplotlib.pyplot as plt

from keras.preprocessing.image import ImageDataGenerator

from keras.utils import load_img, img_to_array

# Preview of the image we will work on

img = load_img('/content/drive/MyDrive/Sample/resized_cat.jpg')

# Rotation

pic = img_to_array(img)

pic_sample = np.expand_dims(pic, axis=0)

data = ImageDataGenerator(rotation_range=50)

aug_data = data.flow(pic_sample, batch_size=1)

fig, axes = plt.subplots(1, 5, figsize=(30, 40))

for i in range(5):

# Accessing each sample

aug_img = next(aug_data)

# Converting to unsigned integer

aug_result = aug_img[0].astype('uint8')

# Showing augmented images

axes[i].imshow(aug_result)

axes[i].axis('off')

Random Brightness

Here, we are randomly increasing and decreasing the brightness of the image by providing brightness_range=[0.5, 1.5].

import numpy as np

import matplotlib.pyplot as plt

from keras.preprocessing.image import ImageDataGenerator

from keras.utils import load_img, img_to_array

# Preview of the image we will work on

img = load_img('/content/drive/MyDrive/Sample/resized_cat.jpg')

# Random Brightness

pic = img_to_array(img)

pic_sample = np.expand_dims(pic, axis=0)

data = ImageDataGenerator(brightness_range=[0.5, 1.5])

aug_data = data.flow(pic_sample, batch_size=1)

fig, axes = plt.subplots(1, 5, figsize=(30, 40))

for i in range(5):

# Accessing each sample

aug_img = next(aug_data)

# Converting to unsigned integer

aug_result = aug_img[0].astype('uint8')

# Showing augmented images

axes[i].imshow(aug_result)

axes[i].axis('off')

Conclusion

Data augmentation is useful when there is less amount of data or is often used for reducing the load of collecting and preparing new data. Many companies see it as a low-cost and effective method to train their AI models.

Data augmentation is more popular with image data than in the NLP domain. The complexity of natural language makes it harder to automate the augmentation of text data.

Let’s point out the topics you’ve learned in this article: