Footnote
The word "learning" may seem rather pretentious for such a simple
formula. However, this is typical of the learning rules used in more complex
neural networks (see Haykin). The
key point is that additional examples can modify the classifier.
With the averaging approach, each additional sample actually has less weight
and is less effective in modifying the classifier. This is appropriate,
unless the classification problem is time varying. If there is a systematic
"drift", the classifier can be made to track this drift by using
a formula such as
.
This amounts to estimating the mean by low-pass filtering
rather than averaging.
We have tacitly assumed that we know the correct class for each of the patterns
used to estimate the mean. This is called supervised learning.
Another way to update a classifier is to use the classifier to classify
a new pattern whose class is unknown, and then use that pattern to update
the mean for the class to which it was assigned. This is called unsupervised
learning or clustering.
Back