🎰
Complete Guide to Machine Learning
  • 👋Preface
  • Chapter 1: Introduction to Machine Learning
    • 1️⃣Defining Machine Learning
    • 2️⃣Types of Machine Learning
    • 3️⃣Applications of Machine Learning
    • 4️⃣Setting up the environment for Machine Learning
  • Chapter 2: Supervised Learning
    • 1️⃣Linear Regression
    • 2️⃣Logistic Regression
    • 3️⃣Decision Trees
    • 4️⃣Random Forest
    • 5️⃣Support Vector Machines
  • Chapter 3: Unsupervised Learning
    • 1️⃣K-Means Clustering
    • 2️⃣Hierarchical Clustering
    • 3️⃣Dimensionality Reduction (PCA, LLE, t-SNE)
  • Chapter 4: Reinforcement Learning
    • 1️⃣Q-Learning
    • 2️⃣SARSA
    • 3️⃣DDPG
  • Chapter 5: Deep Learning
    • 1️⃣Introduction to Neural Networks
  • 2️⃣Convolutional Neural Networks
  • 3️⃣Recurrent Neural Networks
  • 4️⃣Generative Adversarial Networks
  • Chapter 6: Transfer Learning
    • 1️⃣Understanding the concept of transfer learning
  • 2️⃣Pre-trained models and feature extraction
  • 3️⃣Fine-tuning pre-trained models
  • Chapter 7: Evaluation and Deployment
    • 1️⃣Model Evaluation Metrics
  • 2️⃣Model Selection
  • 3️⃣Deployment Strategies
  • Chapter 8: Case Studies and Applications
    • 1️⃣Image Classification
  • 2️⃣Natural Language Processing
  • 3️⃣Recommender Systems
  • Conclusion
    • 1️⃣Summary of main concepts
  • 2️⃣Additional Resources
  • 3️⃣Final project or challenge
Powered by GitBook
On this page
  1. Chapter 2: Supervised Learning

Support Vector Machines

PreviousRandom ForestNextK-Means Clustering

Last updated 2 years ago

Support Vector Machine (SVM) is a type of supervised learning algorithm that can be used for classification or regression problems. The main idea behind SVM is to find the best boundary (or "hyperplane") that separates the different classes in the data. This boundary is chosen in such a way that it maximizes the margin, which is the distance between the boundary and the closest points from each class, also known as support vectors.

SVMs are particularly useful when the data has many features and when the classes are not linearly separable, meaning that a straight line (or a hyperplane in higher dimensions) cannot properly separate the classes. In such cases, SVMs can use a technique called the "kernel trick" to transform the data into a higher-dimensional space where a linear boundary can be applied.

Example

In this example, we first generate some synthetic data using the make_classification function from the sklearn.datasets module. Next, we create an instance of the SVC (support vector classifier) with a linear kernel and a regularization parameter of 1. We then fit the model to the data using the fit method and make predictions on new data using the predict method.

There are different types of kernel that can be used in SVM, such as linear, polynomial, and radial basis function (RBF). The choice of kernel depends on the nature of the data and the problem.

SVM is a powerful algorithm and it's widely used in many industry problem, as it can handle high dimensional data and also it can handle non-linearity by using kernel trick. However, it's worth noting that a SVM model can be overfitting and it can be resolved by tuning the parameters or by using the techniques like cross-validation, pruning and regularization.

Python code


from sklearn import svm
from sklearn.datasets import make_classification

# Generate some synthetic data
X, y = make_classification(n_features=4, n_informative=2, n_redundant=0, random_state=0)

# Create the model
clf = svm.SVC(kernel='linear', C=1)

# Train the model
clf.fit(X, y)


# Generate new data for predictions
X_new, _ = make_classification(n_features=4, n_informative=2,
                           n_redundant=0, random_state=1)

# Predict on new data
y_pred = clf.predict(X_new)
print(y_pred)

Output

[1 0 1 1 1 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 0 0 1 1 0 1 0 1 0
 0 0 1 0 1 1 1 0 0 1 1 1 0 1 1 1 1 1 0 0 1 1 1 0 0 1 1 1 0 1 1 1 1 1 1 0 1
 0 1 0 1 0 0 1 0 0 0 0 1 1 1 1 0 0 0 0 1 0 0 1 0 1 1]

References

https://en.wikipedia.org/wiki/Support_vector_machine

5️⃣
  • Example
  • Python code
LogoMachine-Learning/Chapter_2_SVM.ipynb at main · ariesharry/Machine-LearningGitHub