Automated testing

In the abstract, software testing involves devising a test case (or, more likely, a set of test cases), running the program with the test case, and checking that the performance of the software with the test case as input is as expected. All three aspects of testing can be automated to a greater or lesser extent.

The most straightforward aspect of testing to automate is the execution of test cases. For programs which do not normally interact directly with users (and instead "respond" to the contents of a disk file or instructions passed over a network, or take input as text from a keyboard, it is relatively simple to write small programs (usually called test scripts[?]) to provide the appropriate input. It is somewhat more complex for programs with a GUI to do similarly, but specialised scripting tools for controlling GUI programs exist and are commonly used for performing testing on this class of applications.

Depending on the form of a program's output, in can also be quite straightforward to check whether the output is as expected - for instance, if the program simply produces a stream of text (such as a dynamically generated web page) a simple comparison with expected output is enough to reveal the existence of a problem. This is not always the case - consider the case of a video game where the test is examining that the rendered images "look correct" to the player's eye.

For a large class of programs, however, automated testing techniques are standard practice, and automated testing tools that support them are widely distributed. In many free software projects, test systems are constructed using standard scripting languages which serve the same purpose.

The automated generation of test cases, however, is a much more complex problem, and use of such automatically-generated test cases is consequently much rarer. Research continues into a variety of methodologies for doing so.