Think of that magnet, say in a good compass. Notice how it doesn't just stop but swings back and forth a few times? Or think of a ball bouncing. It doesn't just find the lowest spot then stop. The extra swings or bounces occur while the object is shedding the energy it had to start with. The energy gets lost into sound waves, little thermal jiggles of molecules, etc.
When you get down to the scale of an atom, two things happen differently.
1. If the atom is not in its lowest energy state, i.e. if the electron is really "going around", it can take a while to ditch the excess energy. If the atom is isolated, about the only escape route for the energy is as a photon (or two) of light. In some cases that process is fairly slow.
2. What if the atom is in its lowest possible energy state? Then by definition, it can't drop any lower. However, it turns out that in this case the electrons aren't exactly "going around". States with definite energy turn out not to be changing at all in time, unlike some little particle whizzing around from one place to another.
So perhaps what you're wondering is why the lowest energy state still has kinetic energy, as if the electron were moving, and more potential energy than it would have if the electron were somewhere in the nucleus.
All we can do is refer you to our many other "mediocre" answers on basic quantum mechanics. These objects simply don't have the properties of classical things. Their waves don't have single velocities, but spreads of velocities. They don't have positions, but spreads of positions. The narrower the range of positions, the broader the range of velocities, due to the nature of what wave property corresponds to velocity. So there's a trade-off. States squashed in near the nucleus have low potential energy, but high kinetic energy from the range of velocities. A medium spread (the size of the atom) minimizes the total energy.
Why don't you see this spread with your magnetic compass? It's there, we think, but the effects for big things are too small to notice.
(published on 11/14/10)