You can get the support vectors using clf.support_vectors_.

Plotting the support vectors:

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm

# we create 40 separable points
np.random.seed(0)
X = np.r_[np.random.randn(20, 2) - [2, 2], np.random.randn(20, 2) + [2, 2]]
Y = [0] * 20 + [1] * 20

# fit the model
clf = svm.SVC(kernel='linear', C=1)
clf.fit(X, Y)

# get the separating hyperplane
w = clf.coef_[0]
a = -w[0] / w[1]
xx = np.linspace(-5, 5)
yy = a * xx - (clf.intercept_[0]) / w[1]


margin = 1 / np.sqrt(np.sum(clf.coef_ ** 2))
yy_down = yy - np.sqrt(1 + a ** 2) * margin
yy_up = yy + np.sqrt(1 + a ** 2) * margin

plt.figure(1, figsize=(4, 3))
plt.clf()
plt.plot(xx, yy, 'k-')
plt.plot(xx, yy_down, 'k--')
plt.plot(xx, yy_up, 'k--')

plt.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1], s=80,
            facecolors='none', zorder=10, edgecolors='k')
plt.scatter(X[:, 0], X[:, 1], c=Y, zorder=10, cmap=plt.cm.Paired,
            edgecolors='k')

plt.axis('tight')
x_min = -4.8
x_max = 4.2
y_min = -6
y_max = 6

XX, YY = np.mgrid[x_min:x_max:200j, y_min:y_max:200j]
Z = clf.predict(np.c_[XX.ravel(), YY.ravel()])

# Put the result into a color plot
Z = Z.reshape(XX.shape)
plt.figure(1, figsize=(4, 3))
plt.pcolormesh(XX, YY, Z, cmap=plt.cm.Paired)

plt.xlim(x_min, x_max)
plt.ylim(y_min, y_max)

plt.xticks(())
plt.yticks(())

plt.show()

Answer from seralouk on Stack Overflow
🌐
MIT
web.mit.edu › 6.034 › wwwbob › svm-notes-long-08.pdf pdf
1 An Idiot’s guide to Support vector machines (SVMs) R. Berwick, Village Idiot
infinite number!) • Support Vector Machine · (SVM) finds an optimal · solution · 4 · Support Vector Machine (SVM) Support vectors · Maximize · margin · • SVMs maximize the margin · (Winston terminology: the ‘street’) around the separating hyperplane.
🌐
GeeksforGeeks
geeksforgeeks.org › machine learning › support-vector-machine-algorithm
Support Vector Machine (SVM) Algorithm - GeeksforGeeks
Hyperplane: A decision boundary separating different classes in feature space and is represented by the equation wx + b = 0 in linear classification. Support Vectors: The closest data points to the hyperplane, crucial for determining the hyperplane ...
Published   2 weeks ago
Discussions

python - How to find the support vectors for SVM? - Stack Overflow
Find centralized, trusted content and collaborate around the technologies you use most. Learn more about Collectives ... Bring the best of human thought and AI automation together at your work. Explore Stack Internal ... I'm using the liblinear library to train a linear SVM on my data. I have access to the weights for each class of the trained model. But I need to figure out which training instances are acting as support vectors... More on stackoverflow.com
🌐 stackoverflow.com
Finding support vectors by hand for SVM - Cross Validated
Bring the best of human thought and AI automation together at your work. Explore Stack Internal ... Consider the points $(1, 1)$, $(2, 2)$ and $(3, 3)$, of class $1$, and the points $(4, 5)$, $(5, 7)$, $(6, 5)$ and $(7, 7)$, of class $-1$. Find the support vectors for a hard-margin SVM. More on stats.stackexchange.com
🌐 stats.stackexchange.com
June 17, 2023
Please explain Support Vector Machines (SVM) like I am a 5 year old.
We have 2 colors of balls on the table that we want to separate. http://i.imgur.com/zDBbD.png We get a stick and put it on the table, this works pretty well right? http://i.imgur.com/aLZlG.png Some villain comes and places more balls on the table, it kind of works but one of the balls is on the wrong side and there is probably a better place to put the stick now. http://i.imgur.com/kxWgh.png SVMs try to put the stick in the best possible place by having as big a gap on either side of the stick as possible. http://i.imgur.com/ePy4V.png Now when the villain returns the stick is still in a pretty good spot. http://i.imgur.com/BWYYZ.png There is another trick in the SVM toolbox that is even more important. Say the villain has seen how good you are with a stick so he gives you a new challenge. http://i.imgur.com/R9967.png There’s no stick in the world that will let you split those balls well, so what do you do? You flip the table of course! Throwing the balls into the air. Then, with your pro ninja skills, you grab a sheet of paper and slip it between the balls. http://i.imgur.com/WuxyO.png Now, looking at the balls from where the villain is standing, they balls will look split by some curvy line. http://i.imgur.com/gWdPX.png Boring adults the call balls data, the stick a classifier, the biggest gap trick optimization, call flipping the table kernelling and the piece of paper a hyperplane. More on reddit.com
🌐 r/MachineLearning
40
54
January 5, 2013
Support Vector Machines: How to find the minimum number of support vectors for a machine learning classification problem ?
A minimum of two support vectors ... in the model. This follows from the observation that the margin at each decision boundary must be defined on each side of the dividing hyperplane by the closest data points, which are the support vectors. ... Mohamad M. Awad ... You can use a support vector machine (SVM) when your ... More on researchgate.net
🌐 researchgate.net
8
2
December 27, 2019
🌐
scikit-learn
scikit-learn.org › stable › modules › svm.html
1.4. Support Vector Machines — scikit-learn 1.8.0 documentation
Proper choice of C and gamma is critical to the SVM’s performance. One is advised to use GridSearchCV with C and gamma spaced exponentially far apart to choose good values. ... You can define your own kernels by either giving the kernel as a python function or by precomputing the Gram matrix. Classifiers with custom kernels behave the same way as any other classifiers, except that: Field support_vectors_ is now empty, only indices of support vectors are stored in support_
Top answer
1 of 2
1

You can get the support vectors using clf.support_vectors_.

Plotting the support vectors:

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm

# we create 40 separable points
np.random.seed(0)
X = np.r_[np.random.randn(20, 2) - [2, 2], np.random.randn(20, 2) + [2, 2]]
Y = [0] * 20 + [1] * 20

# fit the model
clf = svm.SVC(kernel='linear', C=1)
clf.fit(X, Y)

# get the separating hyperplane
w = clf.coef_[0]
a = -w[0] / w[1]
xx = np.linspace(-5, 5)
yy = a * xx - (clf.intercept_[0]) / w[1]


margin = 1 / np.sqrt(np.sum(clf.coef_ ** 2))
yy_down = yy - np.sqrt(1 + a ** 2) * margin
yy_up = yy + np.sqrt(1 + a ** 2) * margin

plt.figure(1, figsize=(4, 3))
plt.clf()
plt.plot(xx, yy, 'k-')
plt.plot(xx, yy_down, 'k--')
plt.plot(xx, yy_up, 'k--')

plt.scatter(clf.support_vectors_[:, 0], clf.support_vectors_[:, 1], s=80,
            facecolors='none', zorder=10, edgecolors='k')
plt.scatter(X[:, 0], X[:, 1], c=Y, zorder=10, cmap=plt.cm.Paired,
            edgecolors='k')

plt.axis('tight')
x_min = -4.8
x_max = 4.2
y_min = -6
y_max = 6

XX, YY = np.mgrid[x_min:x_max:200j, y_min:y_max:200j]
Z = clf.predict(np.c_[XX.ravel(), YY.ravel()])

# Put the result into a color plot
Z = Z.reshape(XX.shape)
plt.figure(1, figsize=(4, 3))
plt.pcolormesh(XX, YY, Z, cmap=plt.cm.Paired)

plt.xlim(x_min, x_max)
plt.ylim(y_min, y_max)

plt.xticks(())
plt.yticks(())

plt.show()

2 of 2
1

Let me assume we are talking about libsvm instead of sklearn svc.

The answer can be found in the LIBLINEAR FAQ. In short, you can't. You need to modify the source code.

Q: How could I know which training instances are support vectors?

Some LIBLINEAR solvers consider the primal problem, so support vectors are not obtained during the training procedure. For dual solvers, we output only the primal weight vector w, so support vectors are not stored in the model. This is different from LIBSVM.

To know support vectors, you can modify the following loop in solve_l2r_l1l2_svc() of linear.cpp to print out indices:

    for(i=0; i<l; i++)
    {
        v += alpha[i]*(alpha[i]*diag[GETI(i)] - 2);
        if(alpha[i] > 0)
            ++nSV;
    }

Note that we group data in the same class together before calling this subroutine. Thus the order of your training instances has been changed. You can sort your data (e.g., positive instances before negative ones) before using liblinear. Then indices will be the same.

Top answer
1 of 1
3

In hard-margin SVM, "If the training data is linearly separable, we can select two parallel hyperplanes that separate the two classes of data, so that the distance between them is as large as possible".

The first thing to do is plot the points to see whether linear separability holds.

The colored regions show the convex hulls of the two sets of points. Clearly we don't have to consider any points within their interiors: they cannot serve as support points. Thus, we can ignore altogether.

Because the linear separability is visually evident, we can easily draw some line that separates these classes. The line works, for instance, as shown. Parallel to that line are the lines of the form for numbers Any value of between and will also separate the classes.

The line evidently lies at a distance from class 1 of and a distance from class -1 of Those distances are found by dropping straight down from the vertex to the line or moving straight up from the vertex to the same line. This enables us to connect these vertices of the classes with the path

where the middle edge moves along the line itself.

On the other hand, the dotted segment between and follows the straight path

This path consists of the hypotenuses of two right triangles. Two of their edges are the (vertical) segments projecting the support points to the separating line. Consequently the length of the dotted line is at least as great as the distance between and perpendicular to the separating line.

On the other hand, if we were to construct a perpendicular at any point along the dashed segment, the path would coincide with the dotted line.

The previous inequality, valid for all separating lines, reduces to an equality in this special case.

You can exploit these ideas to prove generally that

the optimal hard-margin hyperplane bisects any line segment that realizes the shortest distance between two linearly separable subsets of

The support vectors can be read off the figure. The distance from this hyperplane to either class is half the distance between the classes.

As an exercise, redo this analysis replacing the point in class 1 with the point As a hint, a line segment realizing the shortest distance now goes from to

(Why did I write "a" line segment? Redo the problem once more upon replacing by the pair of points You should find many line segments that realize the shortest distances between the classes.)


An hour or so spent studying convex analysis -- definitions of convexity, of convex hulls, characterizations and properties of convex sets, and the proofs of equivalence of those characterizations -- will have an immediate payoff for the intuition as well as your understanding of SVM (and of many fundamental ideas of optimization, too).

🌐
Quora
quora.com › How-can-you-find-support-vectors-when-programming-SVM
How to find support vectors when programming SVM - Quora
Answer: It is simple if you are using a quadratic solver(in most cases we use it). The vector returned from quadratic solver be \alpha = (\alpha_1, \alpha_2, ....., \alpha_N) where N \, is the number of training examples used to train SVM. This vector will have non-negative values ( \alpha_i \g...
🌐
Reddit
reddit.com › r/machinelearning › please explain support vector machines (svm) like i am a 5 year old.
r/MachineLearning on Reddit: Please explain Support Vector Machines (SVM) like I am a 5 year old.
January 5, 2013 - Imagine a circus tent that has ... a data point you've trained the SVM with. The fabric is really way up near the pole (support vector), and falls off to the ground on the perimeter....
Find elsewhere
🌐
Medium
medium.com › low-code-for-advanced-data-science › support-vector-machines-svm-an-intuitive-explanation-b084d6238106
Support Vector Machines (SVM): An Intuitive Explanation | by Tasmay Pankaj Tibrewal | Low Code for Data Science | Medium
October 28, 2023 - A journal of articles written by (and for) the KNIME Community around visual programming, data science algorithms & techniques, integration with external tools, case studies, success stories, data processing, and (of course) KNIME Software. ... Support Vector Machines (SVMs) are a type of supervised machine learning algorithm used for classification and regression tasks. They are widely used in various fields, including pattern recognition, image analysis, and natural language processing. ... SVMs work by finding the optimal hyperplane that separates data points into different classes.
Top answer
1 of 8
2
You can use a support vector machine (SVM) when your data has exactly two classes. An SVM classifies data by finding the best hyperplane that separates all data points of one class from those of the other class. The best hyperplane for an SVM means the one with the largest margin between the two classes. The support vectors are the data points that are closest to the separating hyperplane; these points are on the boundary of the slab. So back to your question how to find the minimum number of support vectors for a machine learning classification problem? The data for training is a set of points (vectors) Tj along with their categories Cj. For some dimension D, the Tj ∊ R^D, and the Cj = ±1. The equation of a hyperplane is f(T)=T′β+b=0 where β ∊ R^d and b is a real number. The following problem defines the best separating hyperplane (i.e., the decision boundary). Find β and b that minimize ||β|| such that for all data points (Tj,Cj), Cjf(Tj)≥1. So, the support vectors are the Tj on the boundary, those for which Cjf(Tj)=1. For mathematical convenience, the problem is usually given as the equivalent problem of minimizing ||β||. This is a quadratic programming problem. The optimal solution (ˆβ,ˆb) enables classification of a vector z as follows: class(z)=sign(z′ˆβ+ˆb)=sign(ˆf(z)). ˆf(z) is the classification score and represents the distance z is from the decision boundary.
2 of 8
1
A minimum of two support vectors are required for each decision hyperplane in the model. This follows from the observation that the margin at each decision boundary must be defined on each side of the dividing hyperplane by the closest data points, which are the support vectors.
🌐
Mathworks
nl.mathworks.com › discovery › support-vector-machine.html
What Is a Support Vector Machine? - MATLAB & Simulink
Use a search technique such as Bayesian optimization that uses probabilistic models to find the optimal hyperparameters. Linear SVMs are used for linearly separable data having exactly two classes.
🌐
Byu
axon.cs.byu.edu › Dan › 678 › miscellaneous › SVM.example.pdf pdf
SVM Example
The goals of the course are to ... and weaknesses of neural and machine learning approaches to problem solving. Topics will include ensembles, recurrent networks, support vector machines, and manifold learning....
🌐
DataCamp
datacamp.com › tutorial › svm-classification-scikit-learn-python
Scikit-learn SVM Tutorial with Python (Support Vector Machines) | DataCamp
December 27, 2019 - The distance between the either nearest points is known as the margin. The objective is to select a hyperplane with the maximum possible margin between support vectors in the given dataset. SVM searches for the maximum marginal hyperplane in the following steps:
🌐
Javatpoint
javatpoint.com › machine-learning-support-vector-machine-algorithm
Support Vector Machine Algorithm - Javatpoint
Support Vector Machine Algorithm - Support Vector Machine (SVM) Algorithm with Machine Learning, Machine Learning Tutorial, Machine Learning Introduction, What is Machine Learning, Data Machine Learning, Applications of Machine Learning, Machine Learning vs Artificial Intelligence etc.
🌐
Medium
medium.datadriveninvestor.com › support-vector-machine-svm-382b1f9b0aae
Support Vector Machine (SVM). [1] Introduction and Methodology | by Arneesh Aima | DataDrivenInvestor
December 8, 2021 - Our main aim in SVM is to “maximize the margins”. Data points for class A say positive class and data points for class B say negative class are taken · The data points are plotted on a graph as per the coordinates of axes , say x1 and x2 · By inspection we figure out how many Support Vectors are required for separation of data · All the vectors are augmented with a bias of 1. For example s1 (1 2) becomes s˜1 (1 2 1) We need to find out values of αi for each of the vectors
🌐
Vebuso
vebuso.com › 2020 › 02 › a-top-machine-learning-algorithm-explained-support-vector-machines-svms
A Top Machine Learning Algorithm Explained: Support ...
February 27, 2020 - NCS helps businesses use data platforms and AI to make smarter decisions. We offer tools for data management, AI deployment, and gaining a competitive edge.
🌐
Medium
blog.statsbot.co › support-vector-machines-tutorial-c1618e635e93
Support Vector Machine (SVM) Tutorial | by Abhishek Ghose | Cube Dev
June 19, 2018 - Here’s a simplified version of what SVMs do: Find lines that correctly classify the training data · Among all such lines, pick the one that has the greatest distance to the points closest to it.
🌐
QuantStart
quantstart.com › articles › support-vector-machines-a-guide-for-beginners
Support Vector Machines: A Guide for Beginners | QuantStart
The ability to apply new kernels allows substantial flexibility for the decision boundaries, leading to greater classification performance. $p > n$ - In situations where the number of features for each object ($p$) exceeds the number of training data samples ($n$), SVMs can perform poorly. This can be seen intuitively, as if the high-dimensional feature space is much larger than the samples, then there are less effective support vectors on which to support the optimal linear hyperplanes, leading to poorer classification performance as new unseen samples are added.
🌐
Analytics Vidhya
analyticsvidhya.com › home › beginner’s guide to support vector machine(svm)
SVM | Support Vector Machine | How does SVM work
March 12, 2021 - A hyperplane is a decision boundary that differentiates the two classes in SVM. A data point falling on either side of the hyperplane can be attributed to different classes. The dimension of the hyperplane depends on the number of input features in the dataset. If we have 2 input features the hyper-plane will be a line. likewise, if the number of features is 3, it will become a two-dimensional plane. Support vectors are the data points that are nearest to the hyper-plane and affect the position and orientation of the hyper-plane.
🌐
KDnuggets
kdnuggets.com › 2016 › 06 › select-support-vector-machine-kernels.html
How to Select Support Vector Machine Kernels - KDnuggets
June 13, 2016 - Support Vector Machine kernel selection can be tricky, and is dataset dependent. Here is some advice on how to proceed in the kernel selection process.
🌐
Towards Data Science
towardsdatascience.com › svm-feature-selection-and-kernels-840781cc1a6c
SVM: Feature Selection and Kernels | by Pier Paolo Ippolito | Towards Data Science
December 10, 2021 - Support Vector Machines (SVM) is a Machine Learning Algorithm which can be used for many different tasks (Figure 1). In this article, I will explain the mathematical basis to demonstrate how this algorithm works for binary classification purposes. ... The main objective in SVM is to find the optimal hyperplane to correctly classify between data points of different classes (Figure 2).