There are several usages of the term, which can be used to refer to:
The most common goals in a Computer Architecture revolve around the tradeoffs between cost and performance (i.e. speed), although other considerations, such as size, weight, and power consumption, may be a factor as well.
Generally, cost is held constant, determined by either system or commercial requirements, and speed and storage capacity are adjusted to meet the cost target.
Computer retailers describe the performance of their machines in terms of clock speed (usually in MHz or GHz). This refers to the cycles per second of the main clock of the CPU. However, this metric is somewhat misleading, as a machine with a higher clock rate may not necessarily have higher performance. Modern CPUs can execute multiple instructions per clock cycle, which dramaticallly speeds-up a program. Other factors aid speed, such as the mix of functional units, bus speeds, available memory, and the type and order of instructions in the programs being run.
But, there are also different types of speed. Interrupt latency is the guaranteed maximum response time of the system to an electronic event (e.g. when the disk drive finishes moving some data). This number is affected by a very wide range of design choices. Computers that control machinery usually need low interrupt latencies, because the machine can't, won't or should not wait. For example, computer-controlled anti-lock brakes should not wait for the computer to finish what it's doing - they should brake. Low latencies can often be had very inexpensively.
Benchmarking, tries to take all these factors into account by measuring the time a computer takes to run through a series of test programs. Although benchmarking shows strengths, it may not help one to choose a computer. Often the measured machines split on different measures. For example, one system might handle scientific applications quickly, while another might play popular video games more smoothly.
The general scheme of optimization is to find the costs of the different parts of the computer. In a balanced computer system, the data rate will be constant for all parts of the system, and cost will be allocated proportionally to assure this. The exact form of the computer system will depend on the constraints and goals it was optimized for.
Also see CPU design.