How to Implement Support Vector Machines (SVM) in Python (2023 Edition)

How to Implement Support Vector Machines in Python (2023 Edition)

Support Vector Machines (SVMs) are one of the most powerful and popular algorithms for classification and regression tasks in Machine Learning. They have been used for such a wide range of applications, including finance, healthcare, natural language processing, computer vision, and robotics.

What is a Support Vector Machine?

A Support Vector Machine (SVM) is a supervised learning algorithm that can be used for binary classification, regression and outlier detection. It uses a linear hyperplane to separate data points into two classes by finding the maximum margin between data points of different classes.

How Does an SVM Work?

An SVM works by mapping data points into a high-dimensional feature space so that data points belonging to different classes can be separated with a hyperplane. The hyperplane acts as the decision boundary which can then be used to classify new data points. SVMs use a technique called the kernel trick which allows them to learn nonlinear decision boundaries.

Implementing an SVM in Python

In this section, we will look at how to implement an SVM in Python. We will be using Scikit-learn, a machine learning library for Python, to implement an SVM. Scikit-learn offers a variety of tools and functions to help us with our task.

Step 1: Import Libraries

First, we need to import the relevant libraries to our Python environment. For this tutorial, we will be using Scikit-learn, NumPy and Matplotlib.

import numpy as np
from sklearn.svm import SVC
from sklearn.datasets import make_blobs
from matplotlib import pyplot as plt

Step 2: Generate Dataset

Next, we need to generate a dataset for us to work with. We will be using Scikit-learn's make_blobs function to generate a synthetic dataset with two classes.

X, y = make_blobs(n_samples=100, centers=2, cluster_std=1.5, random_state=0)

plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='autumn')
plt.show()

Step 3: Train the Model

Once we have our dataset, we can train the SVM model. We will be using the SVC class from Scikit-learn for this task. We will first create an instance of the SVC class and then call the fit method to train the model.

model = SVC(kernel='linear')
model.fit(X, y)

Step 4: Visualize the Decision Boundary

Now that we have trained the model, we can visualize the decision boundary. We will be using Matplotlib's contour function to plot the decision boundary.

x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.1),
                     np.arange(y_min, y_max, 0.1))

Z = model.predict(np.c_[xx.ravel(), yy.ravel()])

Z = Z.reshape(xx.shape)

plt.contourf(xx, yy, Z, cmap='autumn', alpha=0.3)
plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='autumn')
plt.show()

Conclusion

In this tutorial, we have seen how to implement an SVM for classification in Python. We have also explored how to visualize the decision boundary. Finally, we have also discussed a few of the advantages and drawbacks associated with SVMs.