In the Otto cycle the fuel and air are mixed outside the cylinder and drawn into it through the intake valve. The mixture is then compressed, and ignited at the right time with a spark plug. This is timed so that the charge is ignited just before the piston reaches the top of the compression stroke, and by the time the combustion is complete it has just started its downward motion. This means that the heating takes place at a constant volume, so the pressure and temperature of the charge increases dramatically.
In the Diesel cycle only air is drawn into the engine and compressed. The fuel is then injected directly into the cylinder with a high-pressure fuel injector when the piston reaches the top of its motion. The fuel is thicker and burns more slowly than petrol, so the piston is already moving down by the time the combustion completes. Thus the Diesel actually loses some of the potential energy of the charge, energy that the Otto cycle captures, although it is less stressful on the engine.
Given that the Diesel is not as efficient, in theory, as an Otto cycle engine, and is also more complex due to the nature of the injectors, it may be surprising that it is used at all. However the key to the Diesel cycle is that the fuel cannot possibly cause the dreaded condition in the Otto cycle known as knock (or ping). Knock occurs when the fuel ignites at the wrong time, typically from heating while it is being compressed, or coming into contact with hot spots inside the cylinder. In order to avoid the problem, Otto cycle engines are often run at lower pressures and temperatures than they are theoretically capable of.
In the Diesel this problem simply can't happen, because the fuel isn't in the engine until the exact moment it is needed. This allows the Diesel to be run at much higher compression ratios, typically double that of an Otto engine of the same size. Since efficiency is strongly related to compression ratio, in practice the Diesel engine is considerably more efficient than a similar power Otto. In fact the Diesel engine has the lowest specific fuel consumption of any large engine, 0.26 lb/hp/hr for very large marine engines.
In addition it is easier to make a slower-burning fuel than petrol, which means that diesel fuel is less expensive to produce and more common. Diesel fuel also has more energy for a given volume than petrol, meaning it takes up less room and longer-range is a common benefit. On the downside, the slower-burning fuel leads to slower maximum RPM, and thus smaller RPM range.
The fuel used in diesel engines is chemically identical to home heating oil. However diesel fuel typically contains a road tax while home heating oil does not, and hence it is generally illegal to use the cheaper heating oil as diesel fuel.
In North America diesel engines are primarily used in large trucks, where the low-stress, high-efficiency cycle leads to much greater engine life and lower operational costs. However cars continue to use gasoline, primarily due to consumer desire for "peppy" cars with a wider range of RPM. In Europe the use of diesel engines is far more common in cars as well. There is considerable interest today in hybrid cars using Diesel engines for most power, with an electric motor to provide the "pep". Current models seem to suggest that cars with all the performance of modern designs can deliver over 100mpg, on the less expensive fuel.
The Diesel cycle is particularly well suited to aviation, where the combination of low specific fuel consumption and low volume make for a smaller aircraft for any given range requirement. Although there was some interest in them in the 1940s, the metallurgy of the era required large blocks to contain the higher compression ratios, which led to them being fairly heavy for their power. They fell from favour after WWII, and from that point on everyone was thinking turbines only. More recently they appear to be making something of a comeback, notably in Europe where avgas is very expensive and hard to find.
Diesel engines are also used in conventional submarines. In these submarines, the diesel engine is run when the submarine is on the surface which charges batteries that power the submarine when it is submerged.