模型假設

Linear Exponential Family

$$f(y;\mu,\theta)=\operatorname{exp}(\frac{y\theta-b(\theta)}{\phi}-S(y,\phi))$$

$$\begin{array}{c|c|c|c|c} & \text{Bin}(n,\pi) \text{ (known } n\text{)} & N(\mu,\sigma^2) & \text{Poisson}(\lambda) & \text{Exp}(\lambda) & \text{Gamma}(\alpha,\lambda) \\ \hline \text{PDF} & \binom{n}{y}\pi^y(1-\pi)^{n-y} & \frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(y-\mu)^2}{2\sigma^2}} & \frac{\lambda^y e^{-\lambda}}{y!} & \lambda e^{-\lambda y} & \frac{\lambda^\alpha}{\Gamma(\alpha)}y^{\alpha-1}e^{-\lambda y} \\ \hline \theta & \operatorname{ln}(\frac{\pi}{1-\pi}) & \mu & \ln\lambda & -\lambda & -\frac{\lambda}{\alpha} \\ \hline b(\theta) & n\operatorname{ln}(1+e^\theta) & \frac{\theta^2}{2} & e^\theta & -\operatorname{ln}(-\theta) & -\operatorname{ln}(-\theta) \\ \hline \phi & 1 & \sigma^2 & 1 & 1 & \frac{1}{\alpha} \\ \hline b'(\theta) & \frac{n}{1+e^{-\theta}}=n\pi & \theta=\mu & e^\theta=\lambda & -\frac{1}{\theta}=\frac{1}{\lambda} & -\frac{1}{\theta}=\frac{\alpha}{\lambda} \\ \hline \phi b''(\theta) & \frac{ne^{-\theta}}{(1+e^{-\theta})^2}=n\pi(1-\pi) & \phi=\sigma^2 & e^\theta=\lambda & \frac{1}{\theta^2}=\frac{1}{\lambda^2} & \frac{1}{\alpha \theta^2}=\frac{\alpha}{\lambda^2} \\ \hline \phi v(\mu) & \mu(1-\frac{\mu}{n}) & \phi & \mu & \mu^2 & \frac{1}{\alpha}\mu^2 \end{array}$$

Canonical Link

$$g(\mu):=\eta=\theta$$

$$\begin{array}{c|c|c|c|c|c} & \text{Bin}(n,\pi) \text{ (known } n\text{)} & N(\mu,\sigma^2) & \text{Poisson}(\lambda) & \text{Exp}(\lambda) & \text{Gamma}(\alpha,\lambda) \\ \hline \theta & \operatorname{ln}(\frac{\pi}{1-\pi}) & \mu & \operatorname{ln}\lambda & -\lambda & -\frac{\lambda}{\alpha} \\ \hline \mu & n\pi & \mu & \lambda & \frac{1}{\lambda} & \frac{\alpha}{\lambda} \\ \hline g(\mu) & \operatorname{ln}(\frac{\mu}{n-\mu}) & \mu & \operatorname{ln}\mu & -\frac{1}{\mu} & -\frac{1}{\mu} \\ \hline \text{Link} & g(\pi)=\operatorname{ln}(\frac{\pi}{1-\pi}) & g(\mu)=\mu & g(\mu)=\operatorname{ln}\mu & g(\mu)=\frac{1}{\mu} & g(\mu)=\frac{1}{\mu} \\ \hline g^{-1}(\eta) & \frac{e^\eta}{1+e^\eta} = \frac{1}{1+e^{-\eta}} & \eta & e^\eta & \frac{1}{\eta} & \frac{1}{\eta} \end{array}$$

模型診斷

模型評價

$$\begin{aligned} Scaled\:Deviance\:\operatorname{D^\ast}&:=2(l_{SAT}-l)\quad \sim \chi^2_{n-p-1} \\ Deviance\: \operatorname{D}&:=\phi \operatorname{D^\ast} \end{aligned}$$

$$\begin{array}{c|c} \text{Dist} & \text{PDF} & l(\mathbf{\mu}) \\\hline \text{Bin}(n,\pi) \text{ (known } n\text{)} & \binom{n}{y}\pi^y(1-\pi)^{n-y} & C+\sum \left[ y_i\operatorname{ln}\mu_i + (n_i-y_i)\operatorname{ln}(n_i-\mu_i) \right] \\ \hline N(\mu,\sigma^2) & \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(y-\mu)^2}{2\sigma^2}} & C - \frac{\sum (y_i-\mu_i)^2}{2\sigma^2} \\ \hline \text{Poisson}(\lambda) & \frac{\lambda^y e^{-\lambda}}{y!} & \sum \left[ -\mu_i + y_i\operatorname{ln}y_i - \operatorname{ln}y_i! \right] \\ \hline \text{Exp}(\lambda) & \lambda e^{-\lambda} & \sum \left( \operatorname{ln}\frac{1}{\mu_i}+\frac{y_i}{\mu_i} \right) \\ \hline \text{Gamma}(\alpha,\lambda) & \frac{\lambda^\alpha}{\Gamma(\alpha)} y^{\alpha-1} e^{-\lambda y} & \text{涉及}\Gamma(\alpha)\text{的計算應該不會考} \end{array}$$

$$\begin{aligned} AIC&=-2l+2 \times (p+1) \\ BIC&=-2l+\operatorname{ln}n \times (p+1) \end{aligned}$$

假設檢定

$$\begin{aligned} H_0 &: \beta_1=\beta_2=...=\beta_r=0 \\ H_a &: not \: all \: =0 \\ \\ LRT&:=2(l_{full}-l_{reduce})= 2(\operatorname{D^\ast_reduce}-\operatorname{D^\ast_full}) \quad \sim\chi^2{r= \Delta p} \end{aligned}$$

殘差分析

$$\begin{aligned} Pearson \: Residuals \: r_i:= \frac{y_i-\hat{\mu_i}}{\sqrt{\hat{\operatorname{Var}}(y_i)}} \end{aligned}$$

$$\begin{aligned} Deviance\: Residuals \: d_i&:= sign(yi-\hat{\mu_i})\times\sqrt{2( \operatorname{ln}f(y_i;\mathbf{\theta_{SAT}})-\operatorname{ln}f(y_i;\hat{\mathbf{\theta_{i}}}) } \\ &=sign(yi-\hat{\mu_i})\times\sqrt{i\operatorname{th} \: term \: of \:\operatorname{D^\ast}} \\\\ &\sum\limits_{i=1}^n (d_i)^2=\operatorname{D^\ast} \end{aligned}$$

Binary & Nominal & Ordinal Regression

Binary

$$\begin{array}{c|c} \text{Link} & g(\pi)=\eta & \pi=g^{-1}(\eta) \\\hline \text{Logit} & \operatorname{ln}(\frac{\pi}{1-\pi}) & \frac{1}{1+e^{-\eta}} \\ \hline \text{Probit} & \Phi^-1(\pi) & \Phi(\eta) \\ \hline \text{Complementary log-log} & \operatorname{ln}(-\operatorname{ln}(1-\pi)) & 1-e^{-e^\eta} \\ \hline \end{array}$$

Logistic Regression

$$\hat{\boldsymbol{\beta}}\:s.t\:\sum^n_{i=1}(y_i-\frac{e^{\mathbf{x'_i\hat{\boldsymbol{\beta}}}}}{1+e^{\mathbf{x'_i\hat{\boldsymbol{\beta}}}}})=0$$

$$\mathbf{I}(\boldsymbol{\beta})=\sum^n_{i=1}\sigma^2_i\mathbf{x_i}\mathbf{x_i}'$$

$$\operatorname{D}= \phi\operatorname{D^*}=2\sum^n_{i=1}[ y_i\operatorname{ln}(\frac{y_i}{\hat{\pi_i}}) + (1-y_i)\operatorname{ln}(\frac{1-y_i}{\hat{1-\pi_i}}) ]$$

Nominal Regression

$$\begin{aligned} g(\pi_i)&=\operatorname{ln}\frac{\pi_j}{\pi_c}=\eta_j \\ \pi_j&=\frac{e^{\eta_j}}{\sum^c_{k=1}e^{\eta_k}} \end{aligned}$$

Ordinal Regression

$$\begin{aligned} \pi_j&:=\operatorname{P}(y=j)\quad j=1,2,3,...c \\ \tau_j&:=\operatorname{P}(y\le j)=\pi_1+\pi_2+...+\pi_j\quad j=1,2,3,...c \end{aligned}$$

Logit Link

$$\begin{aligned} g(\tau_j)&=\operatorname{ln}\frac{\tau_j}{1-\tau_j}=\operatorname{ln}\frac{\pi_1+\pi_2+...+\pi_j}{\pi_{j+1}+\pi_{j+2}+...+\pi_c}=\alpha_j+\eta \\ &(\eta=\mathbf{x'}\boldsymbol{\beta}\text{沒有截距，因為被算到各個}\alpha_j中) \\\\ \tau_j&=\frac{1}{1+e^{-\alpha_j+\eta}} \end{aligned}$$

Probit Link

$$\begin{aligned} g(\tau_j)&=\Phi^{-1}(\tau_j)=\alpha_j+\eta \\ &(\eta=\mathbf{x'}\boldsymbol{\beta}\text{沒有截距，因為被算到各個}\alpha_j中) \\\\ \tau_j&=\Phi(\alpha_j+\eta) \end{aligned}$$

Poisson Regression

$$\hat{\boldsymbol{\beta}}\:s.t\:\sum^n_{i=1}(y_i-\hat{\mu_i})\mathbf{x_i}=0$$

$$\mathbf{I}(\boldsymbol{\beta})=\sum^n_{i=1}\mu_i\mathbf{x_i}\mathbf{x_i}'$$

$$\operatorname{D}= \phi\operatorname{D^*}=2\sum^n_{i=1}[ y_i\operatorname{ln}(\frac{y_i}{\hat{\lambda_i}}) + (y_i-{\hat{\lambda_i}}) ]$$

適合度檢定

$$\begin{aligned} Q_{\text{Poisson}}&=\sum^n_{i=1}\frac{(y_i-\hat{\mu_i})^2}{\hat{\mu_i}} \quad\sim\chi^2_{n-(p+1)} \\ Q_{\text{Binomial}}&=\sum^m_{g=1}\frac{(n_g-n\hat{p_g})^2}{n\hat{p_g}} \quad\sim\chi^2_{(g-1)-(p+1)} \end{aligned}$$

Overdispersion & Underdispersion

賭不會考