TensorFlowJS: Making Predictions from 2D Data
Mohamad Mahmood
How to develop a basic TensorFlowJS project?
The following tutorial is a simplified guide for running the JavaScript version of TensorFlow on CodePen platform based on the original codelab guide https://codelabs.developers.google.com/codelabs/tfjs-training-regression. Refer the original guide for further explanation on the use of the codes.
NOTE: DEEP LEARNING MODEL INVOLVES MANY TERMS AND TECHNIQUES. YOU MAY NEED TO READ THE FOLLOWING STEPS SEVERAL TIMES IN ORDER TO FULLY UNDERSTAND THEM.
[1] Start with the skeleton code
<html>
<head>
<title>TensorFlow.js Tutorial</title>
<!-- Import TensorFlow.js -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@2.0.0/dist/tf.min.js"></script>
<!-- Import tfjs-vis -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-vis@1.0.2/dist/tfjs-vis.umd.min.js"></script>
</head>
<body>
<script>
console.clear();
console.log('#101', 'Hello TensorFlow');
async function getData() {
}
async function run() {
// Part1-Load and plot the original input data that we are going to train on.
// Part2-Convert data to tensor
// Part3-Train the model
// Part4-Predict (and compare the results to the original data)
}
document.addEventListener('DOMContentLoaded', run);
function convertToTensor(data) {}
function createModel() {}
// Create the model
async function trainModel(model, inputs, labels) {}
function testModel(model, inputData, normalizationData) {}
</script>
</body>
</html>
The above code sets up a basic TensorFlow.js application, which includes (1) importing the necessary TensorFlow.js and tfjs-vis libraries, (2) defining several key functions to load and preprocess data, (3) create and train a machine learning model, and (4) evaluate its performance, all orchestrated by an asynchronous run() function that is triggered when the HTML document is fully loaded.
The code demonstrates the core steps involved in building a client-side machine learning application with TensorFlow.js, such as (1) converting data to tensors, (2) defining and training a model, and (3) using the trained model to make predictions, showcasing the flexibility and power of incorporating advanced AI capabilities directly into web-based projects.
CodePen: https://codepen.io/mohamad-razzi/pen/bGyaQxj
[2] Add the codes
[2.1] Get data
async function getData() {
const carsDataResponse = await fetch('https://storage.googleapis.com/tfjs-tutorials/carsData.json');
const carsData = await carsDataResponse.json();
const cleaned = carsData.map(car => ({
mpg: car.Miles_per_Gallon,
horsepower: car.Horsepower,
}))
.filter(car => (car.mpg != null && car.horsepower != null));
console.log('#102', cleaned);
return cleaned;
}
The provided code defines an asynchronous function called getData() that fetches car data from a remote URL, processes the data by extracting and filtering the mpg (Miles per Gallon) and horsepower properties, and returns the cleaned data. The function uses the fetch() API to asynchronously retrieve the data, waits for the response to be converted to JSON, maps over the data to create a new array with the desired properties, filters out any cars with missing values, logs the cleaned data to the console, and finally returns the cleaned data for further use in a machine learning or data analysis context.
[2.2] Add Part1 codes (load data)
Add the following codes to the Part1:
// Part1-Load and plot the original input data that we are going to train on.
const data = await getData();
const values = data.map(d => ({
x: d.horsepower,
y: d.mpg,
}));
tfvis.render.scatterplot({
name: 'Horsepower v MPG'
}, {
values
}, {
xLabel: 'Horsepower',
yLabel: 'MPG',
height: 300
});
The provided code loads and plots the original input data that will be used to train a machine learning model, specifically a scatter plot of car horsepower vs. miles per gallon (MPG). It first calls an asynchronous getData() function to fetch and clean the car data, extracting the horsepower and mpg properties and filtering out any cars with missing values. The cleaned data is then mapped to create an array of {x, y} objects, where x represents horsepower and y represents MPG, which is then used to render a scatter plot visualization using the tfvis.render.scatterplot() function from the TensorFlow.js Visualization library, with the plot labeled with the x-axis as Horsepower and the y-axis as MPG, providing a visual representation of the relationship between the two variables in the input data.
[2.3] Add Part2 codes (convert data to tensor)
Add the following codes to the Part2:
// Part2-Convert data to tensor
const tensorData = convertToTensor(data);
const {
inputs,
labels
} = tensorData;
Add the following function convertToTensor(data):
function convertToTensor(data) {
// Wrapping these calculations in a tidy will dispose any
// intermediate tensors.
return tf.tidy(() => {
// Step 1. Shuffle the data
tf.util.shuffle(data);
// Step 2. Convert data to Tensor
const inputs = data.map(d => d.horsepower)
const labels = data.map(d => d.mpg);
const inputTensor = tf.tensor2d(inputs, [inputs.length, 1]);
const labelTensor = tf.tensor2d(labels, [labels.length, 1]);
//Step 3. Normalize the data to the range 0 - 1 using min-max scaling
const inputMax = inputTensor.max();
const inputMin = inputTensor.min();
const labelMax = labelTensor.max();
const labelMin = labelTensor.min();
const normalizedInputs = inputTensor.sub(inputMin).div(inputMax.sub(inputMin));
const normalizedLabels = labelTensor.sub(labelMin).div(labelMax.sub(labelMin));
return {
inputs: normalizedInputs,
labels: normalizedLabels,
// Return the min/max bounds so we can use them later.
inputMax,
inputMin,
labelMax,
labelMin,
}
});
}
The convertToTensor() function takes in some data, shuffles it, and converts the horsepower and mpg values into separate input and label tensors, respectively, using TensorFlow.js. It then normalizes the input and label tensors to the range of 0 to 1 using min-max scaling, and returns an object containing the normalized tensors, as well as the minimum and maximum values for the input and label data, which can be used later to denormalize the model’s predictions. This data preparation is necessary to ensure the data is in a format that can be efficiently processed by the TensorFlow.js library during the machine learning model’s training and inference stages, and the tf.tidy() function is used to manage memory usage by disposing of any intermediate tensors created during the conversion process.
Normalization
The normalization of data before training a machine learning model is not strictly compulsory, but it is generally considered a good practice as it can provide several benefits. Without normalization, the features in the data may have vastly different scales, which can cause the optimization algorithms used in training to converge much slower, leading to unstable training and making it harder to initialize the model parameters effectively. Additionally, the lack of normalization can make it more difficult to interpret the learned model parameters, as the relative importance of each feature may not be immediately apparent. While it is possible to train a model without normalization, it typically requires more careful tuning of hyperparameters, optimization algorithms, and initialization strategies to overcome these challenges, and normalization helps create a more stable and efficient training process, ultimately leading to better model performance.
[2.4] Add Part3 codes (train the model)
Create the model:
function createModel() {
// Create a sequential model
const model = tf.sequential();
// Add a single input layer
model.add(tf.layers.dense({
inputShape: [1],
units: 1,
useBias: true
}));
// Add an output layer
model.add(tf.layers.dense({
units: 1,
useBias: true
}));
return model;
}
// Create the model
const model = createModel();
console.log('#103', JSON.parse(JSON.stringify(model)));
tfvis.show.modelSummary({
name: 'Model Summary'
}, model);
The provided code defines a function called createModel() that creates a simple linear regression model using TensorFlow.js, which is then instantiated and its summary is displayed in the browser.
The createModel() function creates a sequential model and adds two fully connected tf.layers.dense() layers to it — the first layer has a single input feature and a single output unit, and the second layer is the output layer with a single output unit. The model is then returned, and the main code creates an instance of this model, logs its JSON representation to the console, and uses the TensorFlow.js Visualization library to display a summary of the model’s architecture, including information about its layers, parameters, and output shape.
Fully Connected Layer
A fully connected (or dense) layer is a type of artificial neural network layer where each neuron in the layer is connected to every neuron in the previous layer, allowing the layer to learn complex non-linear relationships between the inputs and outputs; the input to a fully connected layer is typically a 1D vector, and the output is also a 1D vector, where the size of the output vector corresponds to the number of neurons in the layer, and the layer has two sets of parameters — weights that determine the strength of the connections between the input features and the neurons, and biases that determine the activation threshold of the neurons, with fully connected layers commonly used in the final stages of neural network architectures for tasks such as image classification, natural language processing, and general function approximation, as in the provided code where the two tf.layers.dense() layers represent the fully connected layers of the simple linear regression model.
Train the model:
async function trainModel(model, inputs, labels) {
// Prepare the model for training.
model.compile({
optimizer: tf.train.adam(),
loss: tf.losses.meanSquaredError,
metrics: ['mse'],
});
const batchSize = 32;
const epochs = 50;
return await model.fit(inputs, labels, {
batchSize,
epochs,
shuffle: true,
callbacks: tfvis.show.fitCallbacks({
name: 'Training Performance'
},
['loss', 'mse'], {
height: 200,
callbacks: ['onEpochEnd']
}
)
});
}
The provided code defines an asynchronous function called trainModel() that takes a Tensorflow.js model, input data, and label data as parameters, compiles the model for training by specifying the optimizer (Adam), loss function (mean squared error), and metrics (mean squared error) to use, sets the batch size to 32 and the number of training epochs to 50, then calls the model.fit() method to train the model using the provided data, with the training process being visualized using the TensorFlow.js Visualization library, and the function returning the training results asynchronously.
ADAM optimizer*The Adam (Adaptive Moment Estimation) optimizer is a popular optimization algorithm used in training deep neural networks that adaptively adjusts the learning rate for each parameter during the training process by maintaining a moving average of the gradients (the first moment, or the mean) and the squared gradients (the second moment, or the uncentered variance), which allows it to take larger steps for infrequent parameters and smaller steps for frequent parameters, while also incorporating momentum by maintaining a moving average of the gradients to help overcome the problem of oscillations that can occur in stochastic gradient descent and accelerate convergence in the right direction, with the optimizer also employing bias correction to address the initial bias in the moving averages and having relatively few hyperparameters to tune, making it a widely used choice for many deep learning applications, as demonstrated in the provided code where the tf.train.adam() function is used to create an instance of the Adam optimizer that is then passed to the model.compile() method to set up the optimizer for training the model.*
Loss Function
The loss function, which is a crucial component in the training of machine learning models, quantifies the difference between the model’s predictions and the true target values (labels) for the training data, with the goal of the training process being to minimize this loss function, which can take various forms depending on the specific problem being solved and the desired characteristics of the model’s output, such as mean squared error for regression problems where the target variable is a continuous numeric value, cross-entropy loss for classification problems where the target variable is a class label, hinge loss for linear classifiers like Support Vector Machines, and categorical cross-entropy loss for multi-class classification problems, as demonstrated in the provided code where the mean squared error loss function, tf.losses.meanSquaredError, is used for the linear regression model being trained, as the loss function serves as the optimization objective that the training algorithm, such as gradient descent, will aim to minimize in order to produce a more accurate and reliable model.
Metrics (mean squared error)
The “metrics” parameter in the model compilation step refers to the evaluation metric(s) that will be used to assess the performance of the trained model, with the mean squared error (MSE) being the metric used in this case, which is a common regression metric that calculates the average of the squared differences between the model’s predicted values and the true target values, providing a measure of the average magnitude of the errors made by the model where lower MSE values indicate better performance, and including this MSE metric allows the training process to report the model’s MSE on the training data, which can be used to monitor the learning progress and evaluate the model’s performance during and after the training process, as demonstrated in the provided code.
[2.5] Test the model
function testModel(model, inputData, normalizationData) {
const {
inputMax,
inputMin,
labelMin,
labelMax
} = normalizationData;
// Generate predictions for a uniform range of numbers between 0 and 1;
// We un-normalize the data by doing the inverse of the min-max scaling
// that we did earlier.
const [xs, preds] = tf.tidy(() => {
const xsNorm = tf.linspace(0, 1, 100);
const predictions = model.predict(xsNorm.reshape([100, 1]));
const unNormXs = xsNorm
.mul(inputMax.sub(inputMin))
.add(inputMin);
const unNormPreds = predictions
.mul(labelMax.sub(labelMin))
.add(labelMin);
// Un-normalize the data
return [unNormXs.dataSync(), unNormPreds.dataSync()];
});
const predictedPoints = Array.from(xs).map((val, i) => {
return {
x: val,
y: preds[i]
}
});
const originalPoints = inputData.map(d => ({
x: d.horsepower,
y: d.mpg,
}));
tfvis.render.scatterplot({
name: 'Model Predictions vs Original Data'
}, {
values: [originalPoints, predictedPoints],
series: ['original', 'predicted']
}, {
xLabel: 'Horsepower',
yLabel: 'MPG',
height: 300
});
}
The provided testModel() function takes a trained machine learning model, the original input data used to train the model, and the minimum and maximum values used for normalizing the input and output data during training, and generates a visualization that compares the model’s predictions on a range of input values to the original data points, by first extracting the normalization parameters from the normalizationData object, then using tf.tidy() to create a tensor of evenly spaced input values, pass them through the model to obtain the predictions, and un-normalize both the input values and the predictions using the inverse of the min-max scaling applied during training, before creating arrays of the original and predicted data points to be plotted using the tfvis.render.scatterplot() function, which produces a scatter plot visualization that allows for the assessment of the model’s performance by comparing the predicted points to the original data.
Final code:
<html>
<head>
<title>TensorFlow.js Tutorial</title>
<!-- Import TensorFlow.js -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@2.0.0/dist/tf.min.js"></script>
<!-- Import tfjs-vis -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-vis@1.0.2/dist/tfjs-vis.umd.min.js"></script>
</head>
<body>
<script>
console.clear();
console.log('#101', 'Hello TensorFlow');
async function getData() {
const carsDataResponse = await fetch('https://storage.googleapis.com/tfjs-tutorials/carsData.json');
const carsData = await carsDataResponse.json();
const cleaned = carsData.map(car => ({
mpg: car.Miles_per_Gallon,
horsepower: car.Horsepower,
}))
.filter(car => (car.mpg != null && car.horsepower != null));
console.log('#102', cleaned);
return cleaned;
}
async function run() {
// Part1-Load and plot the original input data that we are going to train on.
const data = await getData();
const values = data.map(d => ({
x: d.horsepower,
y: d.mpg,
}));
tfvis.render.scatterplot({
name: 'Horsepower v MPG'
}, {
values
}, {
xLabel: 'Horsepower',
yLabel: 'MPG',
height: 300
});
// Part2-Convert data to tensor
const tensorData = convertToTensor(data);
const {
inputs,
labels
} = tensorData;
// Part3-Train the model
await trainModel(model, inputs, labels);
console.log('#104', 'Done Training');
// Part4-Predict
// Make some predictions using the model and compare them to the
// original data
testModel(model, data, tensorData);
}
document.addEventListener('DOMContentLoaded', run);
function convertToTensor(data) {
// Wrapping these calculations in a tidy will dispose any
// intermediate tensors.
return tf.tidy(() => {
// Step 1. Shuffle the data
tf.util.shuffle(data);
// Step 2. Convert data to Tensor
const inputs = data.map(d => d.horsepower)
const labels = data.map(d => d.mpg);
const inputTensor = tf.tensor2d(inputs, [inputs.length, 1]);
const labelTensor = tf.tensor2d(labels, [labels.length, 1]);
//Step 3. Normalize the data to the range 0 - 1 using min-max scaling
const inputMax = inputTensor.max();
const inputMin = inputTensor.min();
const labelMax = labelTensor.max();
const labelMin = labelTensor.min();
const normalizedInputs = inputTensor.sub(inputMin).div(inputMax.sub(inputMin));
const normalizedLabels = labelTensor.sub(labelMin).div(labelMax.sub(labelMin));
return {
inputs: normalizedInputs,
labels: normalizedLabels,
// Return the min/max bounds so we can use them later.
inputMax,
inputMin,
labelMax,
labelMin,
}
});
}
function createModel() {
// Create a sequential model
const model = tf.sequential();
// Add a single input layer
model.add(tf.layers.dense({
inputShape: [1],
units: 1,
useBias: true
}));
// Add an output layer
model.add(tf.layers.dense({
units: 1,
useBias: true
}));
return model;
}
// Create the model
const model = createModel();
console.log('#103', JSON.parse(JSON.stringify(model)));
tfvis.show.modelSummary({
name: 'Model Summary'
}, model);
async function trainModel(model, inputs, labels) {
// Prepare the model for training.
model.compile({
optimizer: tf.train.adam(),
loss: tf.losses.meanSquaredError,
metrics: ['mse'],
});
const batchSize = 32;
const epochs = 50;
return await model.fit(inputs, labels, {
batchSize,
epochs,
shuffle: true,
callbacks: tfvis.show.fitCallbacks({
name: 'Training Performance'
},
['loss', 'mse'], {
height: 200,
callbacks: ['onEpochEnd']
}
)
});
}
function testModel(model, inputData, normalizationData) {
const {
inputMax,
inputMin,
labelMin,
labelMax
} = normalizationData;
// Generate predictions for a uniform range of numbers between 0 and 1;
// We un-normalize the data by doing the inverse of the min-max scaling
// that we did earlier.
const [xs, preds] = tf.tidy(() => {
const xsNorm = tf.linspace(0, 1, 100);
const predictions = model.predict(xsNorm.reshape([100, 1]));
const unNormXs = xsNorm
.mul(inputMax.sub(inputMin))
.add(inputMin);
const unNormPreds = predictions
.mul(labelMax.sub(labelMin))
.add(labelMin);
// Un-normalize the data
return [unNormXs.dataSync(), unNormPreds.dataSync()];
});
const predictedPoints = Array.from(xs).map((val, i) => {
return {
x: val,
y: preds[i]
}
});
const originalPoints = inputData.map(d => ({
x: d.horsepower,
y: d.mpg,
}));
tfvis.render.scatterplot({
name: 'Model Predictions vs Original Data'
}, {
values: [originalPoints, predictedPoints],
series: ['original', 'predicted']
}, {
xLabel: 'Horsepower',
yLabel: 'MPG',
height: 300
});
}
</script>
</body>
</html>
Equivalent Python codes
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import requests
# Fetch data
def get_data():
response = requests.get('https://storage.googleapis.com/tfjs-tutorials/carsData.json')
cars_data = response.json()
cleaned = [{'mpg': car['Miles_per_Gallon'], 'horsepower': car['Horsepower']}
for car in cars_data if car['Miles_per_Gallon'] is not None and car['Horsepower'] is not None]
print('#102', cleaned)
return cleaned
# Convert data to tensor
def convert_to_tensor(data):
# Shuffle data using numpy
np.random.shuffle(data)
inputs = np.array([d['horsepower'] for d in data], dtype=np.float32)
labels = np.array([d['mpg'] for d in data], dtype=np.float32)
input_tensor = tf.convert_to_tensor(inputs, dtype=tf.float32)
label_tensor = tf.convert_to_tensor(labels, dtype=tf.float32)
input_max = tf.reduce_max(input_tensor)
input_min = tf.reduce_min(input_tensor)
label_max = tf.reduce_max(label_tensor)
label_min = tf.reduce_min(label_tensor)
normalized_inputs = (input_tensor - input_min) / (input_max - input_min)
normalized_labels = (label_tensor - label_min) / (label_max - label_min)
return {
'inputs': normalized_inputs,
'labels': normalized_labels,
'input_max': input_max,
'input_min': input_min,
'label_max': label_max,
'label_min': label_min
}
# Create model
def create_model():
model = tf.keras.Sequential([
tf.keras.layers.Dense(units=1, input_shape=[1], use_bias=True),
tf.keras.layers.Dense(units=1, use_bias=True)
])
return model
# Train model
def train_model(model, inputs, labels):
model.compile(optimizer=tf.optimizers.Adam(),
loss=tf.losses.MeanSquaredError(),
metrics=['mse'])
history = model.fit(inputs, labels, batch_size=32, epochs=50, shuffle=True)
return history
# Test model
def test_model(model, input_data, normalization_data):
input_max = normalization_data['input_max']
input_min = normalization_data['input_min']
label_max = normalization_data['label_max']
label_min = normalization_data['label_min']
# Generate normalized input values for predictions
xs_norm = tf.linspace(0.0, 1.0, 100)
xs_norm_reshaped = tf.reshape(xs_norm, [100, 1]) # Reshape using tf.reshape
predictions = model.predict(xs_norm_reshaped)
# Un-normalize the data
unnorm_xs = xs_norm * (input_max - input_min) + input_min
unnorm_preds = predictions * (label_max - label_min) + label_min
# Prepare data for plotting
predicted_points = np.array([unnorm_xs.numpy(), unnorm_preds.numpy().flatten()]).T # Use .numpy() and .flatten()
original_points = np.array([[d['horsepower'], d['mpg']] for d in input_data])
# Plot the results
plt.scatter(original_points[:, 0], original_points[:, 1], label='Original Data')
plt.plot(predicted_points[:, 0], predicted_points[:, 1], label='Predicted Data', color='red')
plt.xlabel('Horsepower')
plt.ylabel('MPG')
plt.legend()
plt.show()
# Main function
def run():
data = get_data()
tensor_data = convert_to_tensor(data)
model = create_model()
print('#103', model.summary())
train_model(model, tensor_data['inputs'], tensor_data['labels'])
print('#104', 'Done Training')
test_model(model, data, tensor_data)
if __name__ == '__main__':
run()
Colab Notebook:
https://colab.research.google.com/drive/1doJkXnPL9iGNDqjq-46RzxwgHwOnPs90?usp=sharing
Subscribe to my newsletter
Read articles from Mohamad Mahmood directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Mohamad Mahmood
Mohamad Mahmood
Mohamad's interest is in Programming (Mobile, Web, Database and Machine Learning). He studies at the Center For Artificial Intelligence Technology (CAIT), Universiti Kebangsaan Malaysia (UKM).