Linear Regression — Formula Sheet

Ŷ = a + bX

b = [n·ΣXY − ΣX·ΣY] / [n·ΣX² − (ΣX)²]

a = Ȳ − b·X̄

b = Cov(X,Y) / Var(X) = σxy / σx²

b = r · (σy / σx)

If U = X − A and V = Y − B, then b = [n·ΣUV − ΣU·ΣV] / [n·ΣU² − (ΣU)²]

If U = (X − A)/C and V = (Y − B)/D, then b = (D/C) · [n·ΣUV − ΣU·ΣV] / [n·ΣU² − (ΣU)²]

r² = (Coefficient of Determination)

% variation explained = r² × 100

The Method of Least Squares finds the line Ŷ = a + bX that minimises the sum of squared vertical errors: Σ(Yi − Ŷi)² = minimum.

Linear Regression Model

Y = α + βX + ε

α = intercept (value of Y when X = 0), β = regression coefficient (slope), ε = error/disturbance

Normal Equations

ΣY = na + bΣX and ΣXY = aΣX + bΣX²

Regression Coefficient

b = [nΣXY − ΣX·ΣY] / [nΣX² − (ΣX)²]

In any cause-effect relationship

X = Independent / Explanatory Variable (the cause; the variable whose value is known).

The linear regression model is

Y = a + bX + ε.

The best-fitted regression line Ŷ = a + bX is obtained by minimising Σeᵢ² = Σ(yᵢ – ŷᵢ)², the sum of squares of errors.