Q. What is logistic regression?

Logistic Regression models how binary (or multinomial) response variable is related to a set of explanatory variables, which can be discrete and/or continuous.

Binary Logistic Regression

It estimates the probability that a characteristic is present (e.g. estimate probability of "success") given the values of explanatory variables, in this case a single categorical variable; π = Pr (Y = 1|X = x).

Y: binary response variable

X = (X₁, X₂, ..., X_k): a set of explanatory variables which can be discrete, continuous, or a combination

Model

π_i=Pr(Y_i=1|X_i=x_i)=exp(β₀+β₁x_i)/1+exp(β₀+β₁x_i),

or

logit(π_i)=log(π_i1−π_i)=β₀+β₁x_i=β₀+β₁x_i1+…+β_kx_ik

Assumptions

• The data Y₁, Y₂, ..., Y_n are independently distributed, i.e., cases are independent.
• Distribution of Y_i is Bin(n_i, π_i), i.e., binary logistic regression model assumes binomial distribution of the response. The dependent variable typically assumes a distribution from an exponential family (e.g. binomial, Poisson, multinomial, normal,...)
• Assumes linear relationship between the logit of the response and the explanatory variables; logit(π) = β₀ + βX.

Model Fit

• Overall goodness-of-fit statistics of the model: Pearson chi-square statistic (X²),Deviance (G²) and Likelihood ratio test and statistic (ΔG²), and Hosmer-Lemeshow test and statistic
• Residual analysis: Pearson, deviance, adjusted residuals, etc...
• Overdispersion

Parameter Estimation

The maximum likelihood estimator (MLE)

Multinomial Logistic Regression

It models how multinomial response variable Y depends on a set of k explanatory variables, X=(X₁, X₂, ... X_k).