In Euclidean geometry, linear separability is a property of two sets of points. This is most easily visualized in two dimensions (the Euclidean plane) by thinking of one set of points as being colored blue and the other set of points as being colored red. These two sets are linearly separable if there exists at least one line in the plane with all of the blue points on one side of the line and all the red points on the other side. This idea immediately generalizes to higher-dimensional Euclidean spaces if the line is replaced by a hyperplane.

The problem of determining if a pair of sets is linearly separable and finding a separating hyperplane if they are, arises in several areas. In statistics and machine learning, classifying certain types of data is a problem for which good algorithms exist that are based on this concept.

Mathematical definition

-dimensional Euclidean space are linearly separable lies on the other side.

is the normal vector is a scalar.

Equivalently, two sets are linearly separable precisely when their respective convex hulls are disjoint (colloquially, do not overlap).

Examples

Three non-collinear points in two classes ('+' and '-') are always linearly separable in two dimensions. This is illustrated by the three examples in the following figure (the all '+' case is not shown, but is similar to the all '-' case):However, not all sets of four points, no three collinear, are linearly separable in two dimensions. The following example would need two straight lines and thus is not linearly separable:Notice that three points which are collinear and of the form "+ ⋅⋅⋅ ⋅⋅⋅ +" are also not linearly separable.

Number of linear separations

be the number of ways to linearly separate N points (in general position) in K dimensions, thenWhen K. In words, one perceptron.

Linear separability of Boolean functions in n variables

A Boolean function in n variables can be thought of as an assignment of 0 or 1 to each vertex of a Boolean hypercube in n dimensions. This gives a natural division of the vertices into two sets. The Boolean function is said to be linearly separablewhere n is the number of variables passed into the function.

Such functions are also called linear threshold logic, or perceptrons. The classical theory is summarized in, as Knuth claims.

.

It is co-NP-complete to decide whether a Boolean function given in disjunctive or conjunctive normal form is linearly separable.

Threshold logic

A linear threshold logic gate, and otherwise outputs 0.

. It is known thatSee for a literature review.

Support vector machines

Classifying data is a common task in machine learning. Suppose some data points, each belonging to one of two sets, are given and we wish to create a model that will decide which set a new data point will be in. In the case of support vector machines, a data point is viewed as a p-dimensional vector (a list of p numbers), and we want to know whether we can separate such points with a (p1)-dimensional hyperplane. This is called a linear classifier. There are many hyperplanes that might classify (separate) the data. One reasonable choice as the best hyperplane is the one that represents the largest separation, or margin, between the two sets. So we choose the hyperplane so that the distance from it to the nearest data point on each side is maximized. If such a hyperplane exists, it is known as the maximum-margin hyperplane and the linear classifier it defines is known as a maximum margin classifier.

, a set of n points of the form

where the yi is a p-dimensional real satisfying

denotes the dot product the (not necessarily normalized) normal vector.

If the training data are linearly separable, we can select two hyperplanes in such a way that they separate the data and there are no points between them, and then try to maximize their distance.

See also

• Clustering (statistics)

• Hyperplane separation theorem

• Kirchberger's theorem

• Perceptron

• Vapnik–Chervonenkis dimension

References