Curved Boundaries

The linear boundaries produced by a minimum-Euclidean-distance classifier may not be flexible enough. For example, if x₁ is the perimeter and x₂ is the area of a figure, x₁ will grow linearly with scale, while x₂ will grow quadratically. This will "warp" the feature space and prevent a linear discriminant function from performing well.

Solutions:

1. Redesign the feature set (e.g., let x₂ be the square root of the area)
2. Try using Mahalanobis distance, which can produce quadratic decision boundaries (see ahead)
3. Try using a neural network (beyond the scope of these notes; see Haykin)