Libraries for Deep Learning

TensorFlow:

• A comprehensive library for building and deploying machine learning models, particularly deep learning.

• Suitable for both research and production environments, widely used in industry.

Keras:

• A high-level API that simplifies the process of building deep learning models.

• Ideal for beginners and rapid prototyping, often runs on top of TensorFlow.

PyTorch:

• A flexible library that provides dynamic computation graphs, making it easier to build complex models.

• Popular in academic research and for projects requiring custom model architectures.

When choosing a deep learning library, consider the following factors:

• Ease of Use: Libraries like Keras are user-friendly and suitable for beginners, allowing for quick model development.

• Control and Flexibility: If you need more control over the model architecture and training process, consider TensorFlow or PyTorch.

• Performance: Evaluate the library's performance for your specific tasks, especially if you're working with large datasets or complex models.

• Integration: Consider how well the library integrates with other tools and frameworks you plan to use.

• Research vs. Production: If you're focused on research, PyTorch may be preferable due to its dynamic computation graph, while TensorFlow is often favored for production environments.

Building Regression Model Using Keras

Regression predicts a continuous value (e.g., temperature, price, strength). Here, we'll use Keras to predict the house price.

Please go through: https://github.com/omkar-2483/Machine-Learning-Deep-Learning/blob/main/D1-introduction-to-deep-learning-with-keras/regressionWithKeras.ipynb

Different terms used in code:

1. Sequential:

• A class in Keras used to create a linear stack of layers, where each layer has one input tensor and one output tensor.

• Why Use It:

  • Ideal for building simple feedforward neural networks where layers are stacked sequentially.

  • Easy to add layers in order using the .add() method.

2. Dense:

• A class used to create fully connected (dense) layers, where each neuron in the layer is connected to every neuron in the previous layer.

• Why Use It:

  • Most common type of layer in deep learning models.

  • Used for tasks like regression, classification, and more.

• Parameters:

  • units: Number of neurons in the layer.

  • activation: Activation function (e.g., 'relu', 'sigmoid', etc.).

  • input_shape: Shape of the input data (only needed for the first layer).

• compile

    model.compile(optimizer='adam', loss='mean_squared_error', metrics=['mae'])
  

  is used to compile the Keras model, which prepares it for training by specifying:

  1. The optimizer to update weights.

  2. The loss function to measure model performance.

  3. The metrics to evaluate during training and testing.

4. Adam

   Adam (Adaptive Moment Estimation) is an optimization algorithm widely used in deep learning. It combines the benefits of:

   • Momentum: Uses a moving average of past gradients to accelerate convergence.

   • RMSProp: Adapts the learning rate for each parameter based on recent gradient magnitudes.

Key Advantages:

• Automatically adjusts learning rates.

• Efficient and works well for large datasets and high-dimensional data.

• Requires minimal parameter tuning.

5. Loss and metric mae

Loss

• A measure of how well the model's predictions match the actual target values.

• Guides the model during training to minimize errors by adjusting weights.

• Type for Regression: Commonly, mean_squared_error (MSE) is used

  

  Ideal Value:

  • As close to 0 as possible.

  • A smaller loss indicates better model performance.

Mean Absolute Error (MAE)

• The average of the absolute differences between predicted and actual values

  

  • Measures the average magnitude of errors without considering their direction.

    Ideal Value:

    • As close to 0 as possible.

    • A lower MAE means the predictions are more accurate.

Shallow Vs Deep Neural Networks

• Shallow Neural Networks: Typically have one or two hidden layers and take input as vectors.

• Deep Neural Networks: Have three or more hidden layers and can process raw data like images and text, automatically extracting features.

Supervised Learning Models

CNNs

convolutional neural networks (CNNs), which are specialized deep learning algorithms designed for image processing.

Architecture of CNN:

CNNs consist of layers including convolutional, pooling, and fully connected layers.

1. Input Layer: Accepts the input image, usually in the format of (n x m x c), where n and m are the height and width of the image, and c is the number of color channels (1 for grayscale, 3 for RGB).

2. Convolutional Layers: Apply filters (kernels) to the input image to extract features. Each filter slides over the image, performing a dot product to produce feature maps.

   

3. Activation Function (ReLU): Typically follows each convolutional layer, applying the Rectified Linear Unit (ReLU) function to introduce non-linearity by turning negative values to zero.

4. Pooling Layers: Reduce the spatial dimensions of the feature maps. Common types include:

   Max-Pooling: Retains the maximum value from each region.

   Average-Pooling: Computes the average value from each region.

   

5. Fully Connected Layers: After several convolutional and pooling layers, the output is flattened and connected to one or more fully connected layers. These layers combine features to make final predictions.

6. Output Layer: Produces the final output, often using a softmax activation function for classification tasks, providing probabilities for each class.

CNN using Keras: https://github.com/omkar-2483/Machine-Learning-Deep-Learning/blob/main/D1-introduction-to-deep-learning-with-keras/Convolutional-Neural-Networks-with-Keras.ipynb

RNNs

recurrent neural networks (RNNs), which are a type of supervised deep learning model. Unlike traditional models that treat data points as independent, RNNs consider the temporal relationships between data points. They take both the current input and the output from the previous time step into account, making them effective for analyzing sequences, such as text or video scenes.

A popular variant of RNNs is the long short-term memory (LSTM) model, which has been successfully applied in various fields, including image generation and handwriting generation.

Un-supervised Learning Model: Autoencoder

autoencoders, which are unsupervised deep learning models used for data compression.

• Autoencoding: A data compression algorithm where both compression and decompression functions are learned from data.

• Data Specificity: Autoencoders are trained on specific data types, meaning they perform poorly on unrelated data (e.g., trained on cars won't compress buildings well).

• Applications: Common uses include data denoising and dimensionality reduction for visualization.

• Architecture: An autoencoder consists of an encoder that compresses the input and a decoder that reconstructs the original input.

• Restricted Boltzmann Machines (RBMs): A popular type of autoencoder used for tasks like fixing imbalanced datasets and automatic feature extraction.

The article explores popular libraries for deep learning such as TensorFlow, Keras, and PyTorch, each offering unique advantages in terms of ease of use, control, performance, and integration. It highlights considerations for choosing a library based on user needs, such as research versus production environments. The article provides insights into building regression models using Keras, explaining common terms and methodologies. Additionally, it contrasts shallow and deep neural networks, covers supervised learning models like CNNs and RNNs, and delves into unsupervised learning models, specifically autoencoders, and their applications.