Definition
The gradient (often abbreviated to grad) is simply a fancy word for derivative
(which is a fancy enough word already). The gradient usually applies to
vector functions, or functions of more than one variable. Thus, a line
has a gradient, also known as its slope. Using gradient for onevariable
functions is just unnecessarily confusing. The word gradient is also used
to indicate a gradual change (you've probably seen it in Photoshop or
the like) but here we are sticking to the math definition. Of course,
you shall see the relation between the two terms.
Realizing that the gradient is just a derivative for multivariable functions,
let's derive some properties.
First, the gradient of a function is a vector. A derivative
gives us the rate of change of one variable. In order to specify multiple
variables, we need a vector, with one component for each variable in the
function. Thus, a 3variable function has a 3component gradient.
Like the regular derivative, the gradient points in
the direction of greatest increase. Do you want to see why? [**ADD IN
PAGE FOR DERIVATION***]However, with multiple directions, this direction
is no longer simply "forward" or "back". If we have
2 variables, then our 2component gradient can specify any direction on
a plane. Likewise, with 3 variables, we can specify any direction in space.
A Twisted Example
Suppose
we have a magical oven with coordinates written on it and special display
screen. We type in any 3 coordinates, and the display gives us the gradient
of the temperature at that point (it thus has 3 components). Also, we
have a cheap oven: it is unstable, and the temperature varies from point
to point inside the oven. However, we have a little robotic arm that can
move anything inside the oven to any other point. Understand the scenario?
Now suppose we are in need of psychiatric help and put the Pillsbury Dough
Boy inside the oven because we think he would taste good. We place
him in some random point in the oven and we want to cook him as fast as
possible. The gradient can help!
The gradient at any location points in the direction of greatest
increase of a function; in this case our function measures temperature.
So,
the gradient tells in which direction to move the doughboy to get him
to cook even faster. Remember that the gradient does not give us
the coordinates of where to go; it gives us the direction to move.
Thus, we would start out at any random point and check the gradient. We
would follow that direction for a bit and get to a new point, which would
have its own gradient (the new best direction), and so on. Eventually
we would get to the hottest part of the oven and we would stay there.
Pretty soon we'd have our cookies and we'd be good
to go.
Before
you eat those cookies, let's make some observations. First, when we reach
the top (the hottest temperature in the oven), what is the gradient? It
will be zero. Why? There is no direction of greatest increase; every direction
leads to a decrease in temperature. Thus, a zero gradient tells
you to stay put; you are at the max.
Wait... what if there are two maximums? Ah, now we are venturing into
the notsopretty underbelly of the gradient. Finding the maximum in regular
(1 variable) functions involves finding all the places where the derivative
is zero, and testing those points. The reason is that the derivative will
point to local maximums and minimums. The absolute max/min must be tested.
The same applies for the gradient, which is simply a generalization of
the derivative. You must find where the gradient is zero, and test those
points to see which one is the absolute max (both will be local maximums).
You can imagine two hills: the top of each has a zero gradient.
Now that we have cleared that up, enjoy your cookie.
Mathematics
We know the definition of the gradient. The symbol
used is an upsidedown delta (a triangle). I can't seem to find the symbol
in Maple, so we'll have to go without it for now.
Notice how the x component is the partial derivative with respect to x,
and the y component is the partial with respect to y. For a one variable
function, there are no y's at all, so the gradient is simply the derivative.
Check out my links section for some vector
calculus sites for some mathematical examples.
Obvious applications of the gradient are finding the max/min of multivariable
functions. Another less obvious but related application is finding the
maximum of a constrained function (a function whose x and y values have
a limited domain, i.e., they are constrained to lie on a certain circle).
This calls for my boy Lagrange, but all in due time, all in due time...
