From Wikipedia, the free encyclopedia

To comply with Wikipedia's quality standards, this article may need to be rewritten. Please help improve this article. The discussion page may contain suggestions.

In statistics, logistic regression is a model used for prediction of the probability of occurrence of an event. It makes use of several predictor variables that may be either numerical or categories. For example, the probability that a person has a heart attack within a specified time period might be predicted from knowledge of the person's age, sex and body mass index. Logistic regression is used extensively in the medical and social sciences as well as marketing applications such as prediction of a customer's propensity to purchase a product or cease a subscription.

Other names for logistic regression used in various other application areas include logistic model, logit model, and maximum-entropy classifier.

Logistic regression is one of a class of models known as generalized linear models.

Contents 1 Formal mathematical specification 2 Extensions 3 Example 4 See also 5 External links 6 References

Logistic regression analyzes binomially distributed data of the form

where the numbers of Bernoulli trials n_i are known and the probabilities of success p_i are unknown. An example of this distribution is the fraction of seeds (p_i) that germinate after n_i are planted.

The model is then that for each trial (value of i) there is a set of explanatory/independent variables that might inform the final probability. These explanatory variables can be thought of as being in a k vector X_i and the model then takes the form

The logits of the unknown binomial probabilities (i.e., the logarithms of the odds) are modelled as a linear function of the X_i.

Note that a particular element of X_i can be set to 1 for all i to yield an intercept in the model. The unknown parameters β_j are usually estimated by maximum likelihood.

The interpretation of the β_j parameter estimates is as the additive effect on the log odds ratio for a unit change in the jth explanatory variable. In the case of a dichotomous explanatory variable, for instance gender, e^β is the estimate of the odds ratio of having the outcome for, say, males compared with females.

The model has an equivalent formulation as

This functional form is commonly identified as a single-layer perceptron or single-layer artificial neural network. A single-layer neural network computes a continuous output instead of a step function. The derivative of p_i with respect to X = x₁...x_k is computed from the general form:

where f(X) is an analytic function in X. With this choice, the single-layer network is identical to the logistic regression model. This function has a continuous derivative, which allows it to be used in backpropagation. This function is also preferred because its derivative is easily calculated:

Extensions of the model exist to cope with multi-category dependent variables and ordinal dependent variables, such as polytomous regression. Multi-class classification by logistic regression is known as multinomial logit modeling. An extension of the logistic model to sets of interdependent variables is the conditional random field.

Let p(x) be the probability of success when the value of the predictor variable is x. Then let

Algebraic manipulation shows that

where is the odds in favor of success.

If we take a simple example, say p(50) = 2/3, then

So when x = 50, a success is twice as likely as a failure. Or, it can be simply said that the odds are 2 to 1.

• Artificial neural network
• Data mining
• Linear discriminant analysis
• Perceptron
• Probit model
• Variable rules analysis
• Jarrow-Turnbull model

• Web-based logistic regression calculator
• A highly optimized Maximum Entropy modeling package
• MALLET Java library, includes a trainer for logistic models

• Agresti, Alan. (2002). Categorical Data Analysis. New York: Wiley-Interscience. ISBN 0-471-36093-7. 
• Amemiya, T. (1985). Advanced Econometrics. Harvard University Press. ISBN 0-674-00560-0. 
• Balakrishnan, N. (1991). Handbook of the Logistic Distribution. Marcel Dekker, Inc.. ISBN 978-0824785871. 
• Green, William H. (2003). Econometric Analysis, fifth edition. Prentice Hall. ISBN 0-13-066189-9. 
• Hosmer, David W.; Stanley Lemeshow (2000). Applied Logistic Regression, 2nd ed.. New York; Chichester, Wiley. ISBN 0-471-35632-8. 

v • d • e Statistics
Descriptive statistics	Mean (Arithmetic, Geometric) - Median - Mode - Power - Variance - Standard deviation
Inferential statistics	Hypothesis testing - Significance - Null hypothesis/Alternate hypothesis - Error - Z-test - Student's t-test - Maximum likelihood - Standard score/Z score - P-value - Analysis of variance
Survival analysis	Survival function - Kaplan-Meier - Logrank test - Failure rate - Proportional hazards models
Probability distributions	Normal (bell curve) - Poisson - Bernoulli
Correlation	Confounding variable - Pearson product-moment correlation coefficient - Rank correlation (Spearman's rank correlation coefficient, Kendall tau rank correlation coefficient)
Regression analysis	Linear regression - Nonlinear regression - Logistic regression