In
statistics the
linear model can be expressed by saying
where
Y is an
nx1 column vector of random variables,
X is an
nxp matrix of "known" (i.e., observable and non-random) quanitities, whose rows correspond to
statistical units,
β is a
px1 vector of (unobservable) parameters, and
ε is an
nx1 vector of "errors", which are uncorrelated random variables each with expected value 0 and variance
σ^{2}. Often one takes the components of the vector of errors to be
independent and
normally distributed. Having observed the values of
X and
Y, the statistician must estimate
β and
σ^{2}. Typically the parameters
β are estimated by the method of
least squares.
If, rather than taking the variance of ε to be σ^{2}I, where I is the nxn identity matrix, one assumes the variance is σ^{2}M,
where M is a known matrix other than the identity matrix, then one estimates β by the method of "generalized least squares", in which, instead of minimizing the sum of squares of the residuals, one minimizes a different quadratic form in the residuals -- the quadratic form being the one given by the matrix M^{-1}.
If all of the off-diagonal entries in the matrix M are 0, then one normally estimates β by the method of "weighted least squares", with weights proportional to the reciprocals of the diagonal entries.
Ordinary Linear regression is a very closely related topic.
"Generalized linear models", rather than saying
say
where
f is the "link function". An example is the "Poisson regression model", which says
- Y_{i} has a Poisson distribution with expected value e^{γ+δxi}.
The link function is the natural logarithm function.
Having observed
x_{i} and
Y_{i} for
i=1,...,n, one can estimate γ and δ by the method of
maximum likelihood.
All Wikipedia text
is available under the
terms of the GNU Free Documentation License