These five pieces were initially quite separate, but it was the elder Tolkien's express wish that they be published together. The Silmarillion, together with other posthumous collections of Tolkien's works, such as Unfinished Tales and The History of Middle-earth, forms a narrative describing the history of the universe where The Hobbit and The Lord of the Rings take place. It is much less detailed than those two works, but it covers a much longer period of time. Although reading The Silmarillion is not necessary to enjoy these books, one can learn much more about Tolkien's world and its concepts by reading it.
The Silmarillion is a very complex work, employing an extremely wide array of themes that originate in lore of countries all over Europe, but not adhering to any of them. Thus, the title of Eru Ilúvatar (One who is Father of All) is clearly borrowed from Norse mythology; the character himself resembles the God of the Bible, and even the writing style in which the Ainulindalë is told resembles that of the Bible; the story of Túrin Turambar is very similar to a motif from Finnish Kalevala; and Númenor is obviously reminiscent of Atlantis (in fact, one of the names Tolkien gave that land was Atalantë, though he gave it an Elvish derivation).
Historically, the first drafts of The Silmarillion stories date back to as early as 1917, when Tolkien was hospitalized in a field hospital with trench fever. He tried to publish some the stories (in a very early version) some time in the 1920s; however most editors regarded them with suspicion (the fairy tale for adults was not a popular concept then). He tried once more, having published The Hobbit in 1937; however that time too, The Silmarillion was found to be too complicated, and so Tolkien was asked to write a simple sequel instead (that sequel developed into The Lord of the Rings).
However Tolkien never abandoned these stories, probably seeing in them the genesis of Middle-earth as it is, the later events (The Hobbit and The Lord of the Rings) being only the aftershocks. The last drafts of the Silmarillion stories were written just a short while before Tolkien's death in 1973. For several years, Christopher Tolkien worked on deciphering and connecting his father's drafts, which often were mere sketches. On some of the later parts of the "Quenta Silmarillion" which were in the roughest state, he worked with fantasy author Guy Gavriel Kay to construct a narrative practically from scratch. The final result, which was connected in a chronological sequence and made consistent, was published in 1977.
During the 1980s and 1990s, Christopher Tolkien published all of his father's writings as the 12-volume History of Middle-earth series. In addition to the source material and earlier drafts of several portions of The Lord of the Rings, these books greatly expand on the original material published in The Silmarillion, and in many cases diverge from it. These later books also reveal that Tolkien developed certain parts of the story of The Silmarillion more than others. The chapters of the story with the greatest (and most interesting) detail include: