First, a ferromagnetic domain is a region in which the magnetism of the individual particles mostly lines up in the same direction.
The reason that the domains can stay stuck for a long time is that certain directions are easier (lower energy) than others. It's always equally easy to point one direction or its opposite, but the intermediate directions can require more energy. It can take a very rare event for a domain in a "hard" magnet to get enough energy to flip through that intermediate high-energy state. In a "soft" magnet (like pure iron) the energy required can be quite low, so the domains don't stay stuck.
The name of this effect (different energies along different magnetic directions) is "magnetic anisotropy". It can come either from the basic properties of the crystal or from the uneven distribution of defects in the crystal. Most good permanent magnets use alloys, in which the random locations of the different atoms makes a good deal of local anisotropy.
Mike W.
(published on 07/13/2012)