The Math Behind ML : Vectors - Operations

In the previous blog, we explored the types of vectors in both mathematics and machine learning. In this post, let’s dive into vector operations and how they play out in real-world scenarios using Python.

Defining vectors in python : [ Row vector and column vector ]

import numpy as np
row_vector=np.array([1,2,3])
column_vector = np.array([[4],[5],[6]])

print(row_vector)
[1,2,3]

print(column_vector)
[[4]
 [5]
 [6]]

Vector Operations Explained With Diagrams :

Vector Addition :

• Vector addition combines two vectors by adding their corresponding components, resulting in a new vector. This can be thought of as extending or shifting the original vector in the direction of the other.

  

import numpy as np
row_vector=np.array([1,2,3])
row_vector_2 = np.array([2,3,4])
print (row_vector+row_vector_2)

Vector Subtraction :

• Vector subtraction finds the difference between two vectors by subtracting their corresponding components. This operation gives a vector that points from the second vector to the first and can be interpreted as a relative displacement.

vec_sub = row_vector-row_vector_2

Scalar Multiplication :

• Scalar multiplication scales a vector by a constant value. It stretches (if scalar > 1), compresses (if 0 < scalar < 1), or reverses direction (if scalar < 0) of the original vector without changing its direction (except sign).

scal_mul = 2*row_vector_2 # 2 is scalar

Dot Product (Scalar Product) :

• The dot product (also called scalar product) of two vectors returns a scalar. It measures how much two vectors point in the same direction.

np_dot = np.dot(row_vector,row_vector_2) # dot product in numpy can be achieved using np.dot(x,y)

# For complex numbers 
a = 3 + 1j
b = 7 + 6j

print(np.dot(a,b))

Cross Product (Only for 3d Vectors) :

• Only defined in 3D, the cross product returns a vector perpendicular to both input vectors.

np_cross = np.cross(row_vector,row_vector_2) # cross product in numpy can be achieved using np.cross(x,y)

Magnitude(Norm) :

• The norm of a vector is its length. Also called magnitude.

vec_norm = np.linalg.norm(row_vector)

Normalization :

• The process of converting a vector to a unit vector (length = 1) while keeping its direction.

    def normalize_vector(v):
        norm = np.linalg.norm(v)
        if norm == 0:
            return v
        return v / norm
  
    normalized_vec = normalize_vector(row_vector)
    print(normalized_vec)
  

Projection :

• The projection of vector an onto vector b is a new vector that represents how much of a goes in the direction of b. One real world example would be changing the view in 3d modeling software like Blender.

def vector_projection(u, v):
  if np.all(v == 0):
    raise ValueError("Cannot project onto a zero vector")
  dot_product = np.dot(u, v)
  v_norm_squared = np.dot(v, v)
  projection = (dot_product / v_norm_squared) * v
  return projection

u = np.array([1, 2, 3])
v = np.array([5, 6, 2])

proj_of_u_on_v = vector_projection(u, v)
a = np.array([2, 1])
b = np.array([4, 0])
proj_of_a_on_b = vector_projection(a, b)

In the next blog, we’ll dive into Vector space and see how these operations form the building blocks of deeper algebraic structures like groups and fields.