Encyclopedia > Linear model

  Article Content

Linear model

In statistics the linear model can be expressed by saying
  • Y=Xβ+ε
where Y is an nx1 column vector of random variables, X is an nxp matrix of "known" (i.e., observable and non-random) quanitities, whose rows correspond to statistical units, β is a px1 vector of (unobservable) parameters, and ε is an nx1 vector of "errors", which are uncorrelated random variables each with expected value 0 and variance σ2. Often one takes the components of the vector of errors to be independent and normally distributed. Having observed the values of X and Y, the statistician must estimate β and σ2. Typically the parameters β are estimated by the method of least squares.

If, rather than taking the variance of ε to be σ2I, where I is the nxn identity matrix, one assumes the variance is σ2M, where M is a known matrix other than the identity matrix, then one estimates β by the method of "generalized least squares", in which, instead of minimizing the sum of squares of the residuals, one minimizes a different quadratic form in the residuals -- the quadratic form being the one given by the matrix M-1. If all of the off-diagonal entries in the matrix M are 0, then one normally estimates β by the method of "weighted least squares", with weights proportional to the reciprocals of the diagonal entries.

Ordinary Linear regression is a very closely related topic.

"Generalized linear models", rather than saying

  • E(Y)=Xβ,
  • f(E(Y))=Xβ,
where f is the "link function". An example is the "Poisson regression model", which says
  • Yi has a Poisson distribution with expected value eγ+δxi.
The link function is the natural logarithm function. Having observed xi and Yi for i=1,...,n, one can estimate γ and δ by the method of maximum likelihood.

All Wikipedia text is available under the terms of the GNU Free Documentation License

  Search Encyclopedia

Search over one million articles, find something about almost anything!
  Featured Article
Wheatley Heights, New York

... Latino of any race. There are 1,455 households out of which 43.6% have children under the age of 18 living with them, 62.2% are married couples living together, 17.0% have ...

This page was created in 26.5 ms