Supervised learning

Supervised learning is a machine learning technique for creating a function from training data. The training data consists of pairs of input objects (typically vectors), and desired outputs. The output of the function can be a continuous value (called regression), or can predict a class label of the input object (called classification). The task of the supervised learner is to predict the value of the function for any valid input object after having seen only a small number of training examples (i.e. pairs of input and target output). To achieve this, the learner has to generalize from the presented data to unseen situations in a "reasonable" way (see inductive bias). (Compare with unsupervised learning.)

In order to solve a given problem of supervised learning (e.g. learning to recognize handwriting) one has to consider various steps:

1. Determine the type of training examples. Before doing anything else, the engineer should decide what kind of data is to be used as an example. For instance, this might be a single handwritten character, a entire handwritten word, or a entire line of handwriting.
2. Gathering a training set. The training set needs to be characteristic of the real-world use of the function. Thus, a set of input objects is gathered and corresponding outputs are also gathered, either from human experts or from measurements.
3. Determine the input feature representation of the learned function. The accuracy of the learned function depends strongly on how the input object is represented. Typically, the input object is transformed into a feature vector, which contains a number of features that are descriptive of the object. The number of features should not be too large, because of the curse of dimensionality[?]; but should be large enough to accurately predict the output.
4. Determine the structure of the learned function and corresponding learning algorithm. For example, the engineer may choose to use neural networks or decision trees.
5. Complete the design. The engineer then runs the learning algorithm on the gathered training set. Parameters of the learning algorithm may be adjusted by optimizing performance on a subset (called a validation set) of the training set, or via cross-validation[?]. After parameter adjustment and learning, the performance of the algorithm may be measured on a test set that is separate from the training set.

Approaches and algorithms

• analytical learning[?]
• artificial neural networks
• backpropagation
• boosting
• Bayesian statistics[?]
• case-based reasoning
• decision tree learning
• inductive logic programming
• learning automata theory[?]
• naive Bayesian classification
• pattern recognition
• probably approximately correct[?] (PAC) learning
• symbolic machine learning[?] algorithms
• subsymbolic machine learning[?] algorithms
• support vector machines

Applications

• bioinformatics
• handwriting recognition
• information retrieval
• object recognition in computer vision
• optical character recognition
• spam detection
• speech recognition

General issues

• computational learning theory[?]
• inductive bias
• overfitting
• version spaces[?]