Statistics is a branch of applied mathematics that includes planning, summarizing, and interpreting uncertain observations. Probability theory plays a critical role in the development of statistical theory. The word comes from the modern Latin phrase statisticum collegium (lecture about state affairs), from which came the Italian word statista, which means "statesman" or "politician" (compare to status) and the German Statistik, originally designating the analysis of data about the state. It acquired the meaning of the collection and classification of data generally in the early nineteenth century.
We describe our knowledge (and ignorance) mathematically and attempt to learn more from whatever we can observe. This requires us to
In some forms of descriptive statistics, notably data mining, the second and third of these steps become so prominent that the first step (planning) appears to become less important. In these disciplines, data often are collected outside the control of the person doing the analysis, and the result of the analysis may be more an operational model than a consensus report about the world.
The probability of an event is often defined as a number between one and zero rather than a percentage. In reality however there is virtually nothing that has a probability of 1 or 0. You could say that the sun will come up in the morning, but what if an extremely unlikely quantum event destroys the sun? What if there is a nuclear war and the sky is covered in ash and smoke?
We often round the probability of such things up or down because they are so likely or unlikely to occur, that it's easier to recognise them as a probability of one or zero.
However, this can often lead to misunderstandings and dangerous behaviour, because people are unable to distinguish between, e.g., a probability of 10-4 and a probability of 10-9, despite the very practical difference between them. If you expect to cross the road about 105 or 106 times in your life, then reducing your risk per road crossing to 10-9 will make you safe for your whole life, while a risk per road crossing of 10-4 will make it very likely that you will have an accident, despite the intuitive feeling that 0.01% is a very small risk.
Statistics form a key basis tool in business and manufacturing as well. It is used to understand measurement systems variability, control processes (as in "statistical process control" or SPC), for summarizing data, and to make data driven decisions. In these roles it is a key tool, and perhaps the only reliable tool.