A rocket is any device that propels itself using reaction mass, please see Newton's 2nd Law of Motion.
Rockets are commonly used when it is necessary to carry all the fuel a vehicle needs (such as in outer space) and there is no other substance (land, water, or air) that a vehicle may push itself with. There are many different types of rockets, and a comprehensive list can be found in spacecraft propulsion.
Most current rockets are chemical rockets. A chemical rocket engine may use solid fuel, like the Space Shuttle's SRBs, or liquid fuel, like the Space Shuttle's main engines. A chemical reaction is initiated with the fuel in the combustion chamber, and the hot gasses are forced out the rear of the rocket. This causes the rocket itself to be thrust forward.
Nuclear-powered thermal rockets have also been developed, but never put into use.
Rockets were first developed by the Chinese a long time ago (how long?), using gunpowder. These were initially developed for entertainment, the precursors to modern firework but were soon adapted for warfare. Because the pressures on the rocket walls are lower, the use of rockets in warfare preceded the use of the gun, which required a higher level of metal technology.
In military terminology, a rocket is generally solid-fueled and unguided. These are typically fired by ground-attack aircraft at fixed targets such as buildings. Whereas a missile can be either solid or liquid-fueled, and has a guidance system.