Key Machine Learning Concepts: Impact on Bias, Variance, Overfitting, and Accuracy💯

1. Model Complexity

Model Complexity ⬆️
   Bias ⬇️
   Variance ⬆️
   Overfitting ⬆️
   Accuracy ⬇️ (if too complex)

2. Regularization

Regularization Strength ⬆️
   Model Complexity ⬇️
   Bias ⬆️
   Variance ⬇️
   Overfitting ⬇️
   Accuracy ⬇️ (if too strong)

3. Learning Rate in Gradient Descent

Learning Rate ⬆️
   Convergence Speed ⬆️
   Stability ⬇️ (risk of overshooting)
   Accuracy ⬇️ (if too high)

Learning Rate ⬇️
   Convergence Speed ⬇️
   Stability ⬆️
   Accuracy ⬇️ (if too low, may not converge)

4. Number of Features

Number of Features ⬆️
   Model Complexity ⬆️
   Bias ⬇️
   Variance ⬆️
   Overfitting ⬆️
   Accuracy ⬇️ (if too many irrelevant features)

5. Training Data Size

Training Data Size ⬆️
   Bias ⬇️
   Variance ⬇️
   Overfitting ⬇️
   Accuracy ⬆️

6. Ensemble Size (in Bagging or Random Forest)

Ensemble Size ⬆️
   Variance ⬇️
   Overfitting ⬇️
   Accuracy ⬆️
   Computational Cost ⬆️

7. Cross-Validation

Cross-Validation Folds ⬆️
   Variance ⬇️
   Overfitting ⬇️
   Accuracy ⬆️ (more reliable estimate)
   Training Time ⬆️

8. Polynomial Degree

Polynomial Degree ⬆️
   Model Complexity ⬆️
   Bias ⬇️
   Variance ⬆️
   Overfitting ⬆️
   Accuracy ⬇️ (if too complex)

9. Feature Engineering

Feature Engineering Quality ⬆️
   Model Complexity ⬆️ (if more features)
   Bias ⬇️
   Variance ⬆️ (if too many features)
   Overfitting ⬆️ (if irrelevant features added)
   Accuracy ⬆️ (if relevant features added)

10. Batch Size in Training

Batch Size ⬆️
   Training Time ⬆️
   Variance ⬇️ (more stable gradients)
   Overfitting ⬇️ (if larger batches)
   Accuracy ⬆️ (if optimal batch size)

Batch Size ⬇️
   Training Time ⬇️
   Variance ⬆️ (more noisy gradients)
   Overfitting ⬆️ (if too small)
   Accuracy ⬇️ (if too small)

11. Dropout Rate in Neural Networks

Dropout Rate ⬆️
   Model Complexity ⬇️ (fewer active neurons)
   Bias ⬆️ (if too high)
   Variance ⬇️
   Overfitting ⬇️
   Accuracy ⬇️ (if too high)

12. Hyperparameter Tuning

Hyperparameter Tuning ⬆️
   Model Performance ⬆️
   Overfitting ⬇️ (if tuned correctly)
   Accuracy ⬆️
   Training Time ⬆️ (more iterations)

13. Data Preprocessing (Normalization/Standardization)

Data Preprocessing Quality ⬆️
   Model Complexity ⬇️ (more stable training)
   Bias ⬇️
   Variance ⬇️
   Overfitting ⬇️
   Accuracy ⬆️

14. Early Stopping

Training Iterations ⬆️
   Overfitting ⬆️ (if training continues too long)
   Accuracy ⬇️ (if overfitting occurs)

Early Stopping ⬆️
   Overfitting ⬇️
   Accuracy ⬆️ (more generalization)

---