A clear understanding of how the brain works will make it possible for us to build intelligent machines, in silicon, that will exceed our human ability in surprising ways. The brain is not a computer, but a memory system that stores experiences in a way that reflects the true structure of the world, remembering sequences of events and their nested relationships and making predictions based on those memories. It is this memory-prediction system that forms the basis of intelligence, perception, creativity, and even consciousness. The neocortex is the seat of most aspects of perception and high-level thought. I propose that the neocortex can be understood as a hierarchical sequence memory. This talk will describe the anatomy and physiology of the cortex, and show that they can be accurately captured by a mathematical model based on conditional probabilities. An implementation of this system will be shown that demonstrates robust invariant image recognition.