I can't imagine it would help matters any to soak fertilizer in olive oil. I'll assume the fertilizer was a dry substance before it got soaked (some are sold as liquids). The reason for this is that plants absorb water-soluble nutrients from the soil and transport them along with water to the places in the plant where they are needed. Soaking pellets or powder of fertilizer in oil may keep it from getting in the ground in the first place, since the fertilizer will either no longer dissolve in water if it is coated in oil or dissolve much more slowly. In the second case, if you want to have a slow time-release of your fertilizer, olive (or other low-volatility) oil may be a good substance to add to achieve that end. Other vegetable oils may be less expensive ways to do this.
That having been said, a very famous combination should be avoided! If the fertilizer is ammonium nitrate, I would recommend keeping all oils away from it. Adding fuel oil to ammonium nitrate makes a highly explosive mixture -- this is what was used to bomb the Murrah federal building in Oklahoma city, and is even used legitimately as an explosive in mining and quarrying operations. Olive oil may be less dangerous as it is less flammable, but I wouldn't know enough to bet my personal safety on it.
(published on 10/22/2007)