Linux Device Drivers for Embedded Systems: Understanding Spinlocks

Before delving into the topic, let's start with some basics.

A resource is considered shared when it is accessed by several contenders. These contenders, such as processors, processes, or threads, may own the resource either exclusively or concurrently.

Operating systems perform mutual exclusion by atomically modifying a variable that holds the current state of a resource. This variable is visible to other contenders, which might access it at the same time. This atomicity guarantees that the modification of this variable is either successful or fails.

The Linux kernel has two synchronization mechanisms:

Locks
Conditional Variables

Locks are essentially hardware mechanisms that allow synchronization through atomic operations. The kernel uses locking facilities as synchronization primitives, which are data structures used to access shared resources. Only one contender can hold the lock at a time, thus controlling the shared resources. This ensures that any operations performed on the shared resources by the lock holder appear atomic to other contenders.

Now come straight to the point. i.e

Spinlocks

A spinlock is a hardware-based locking primitive that relies on the capabilities of the hardware to provide atomic operations. Spinlocks are used in atomic contexts where sleeping is not allowed or not needed, such as in interrupt handlers or when preemption needs to be disabled. They also serve as an inter-CPU locking primitive, ensuring mutual exclusion across multiple processors.

Let us understand this with the diagram below:

Figure 1: Spinlock Working Flow

Let’s explore the diagram

A CPU (e.g., CPU B) running a task (e.g., Task B) tries to acquire a spinlock.
The spinlock checks if the lock is already held by another CPU (e.g., CPU A running Task A).
If the lock is held by another CPU, CPU B enters a spinning state.
In this spinning state, CPU B repeatedly checks if the lock has been released (this involves a busy-wait loop).
CPU B continuously tests the lock in a tight loop to see if the lock becomes available.
This spinning prevents CPU B from performing other tasks, as it is busy checking the lock status.
When CPU A finishes its critical section, it releases the spinlock by calling the spinlock’s release function.
Once the spinlock is released, CPU B detects that the lock is now available.
CPU B then successfully acquires the spinlock and proceeds with its critical section.
While a task holds a spinlock, the CPU running that task is not preempted by other tasks (scheduler is disabled for that CPU).

Some important notes from above execution
The CPU can still be interrupted by IRQs (Interrupt Requests) if they are not disabled. On a single-core machine, spinlocks are less relevant because the task holding the spinlock cannot be preempted by another task, effectively achieving the same result without spinning. Spinlocks are suited for scenarios where the critical section is short and the overhead of spinning is less than the overhead of context switching.

This approach ensures that only one CPU accesses a shared resource at a time, maintaining data consistency in a multi-core environment.

So far, we understand how spinlocks work, right?

Let's move on to the practical aspects of creating spinlocks. There are two ways to create spinlocks:

Define spinlock macros: This method involves using predefined macros to create and initialize spinlocks.
Dynamically callingspin_lock_init()function: This method involves dynamically initializing a spinlock using the spin_lock_init() function.

First, we'll explore using the DEFINE_SPINLOCK macro. To understand this in detail, let's examine its definition in include/linux/spinlock_types.h (see source code):

#define DEFINE_SPINLOCK(x) spinlock_t x = __SPIN_LOCK_UNLOCKED(x)

This can be used as follows:

static DEFINE_SPINLOCK(foo_lock);

After this, the spinlock will be accessible through its name, foo_lock.

For dynamic (runtime) allocation, you need to embed the spinlock into a structure, allocate memory for this structure, and then call spin_lock_init() on the spinlock element:

struct dummy_struct {
    spinlock_t lock;
    unsigned int foo;
    // Other members...
};

static struct dummy_struct *demo_alloc_init_function() {
    struct dummy_struct *ds;
    ds = kmalloc(sizeof(struct dummy_struct), GFP_KERNEL);
    if (!ds)
        return -ENOMEM;
    spin_lock_init(&ds->lock);
    return bs;
}

Important tip: It is better to use DEFINE_SPINLOCK whenever possible. It offers compile-time initialization and requires fewer lines of code with no real drawbacks. At this stage, we can lock and unlock the spinlock using the spin_lock() and spin_unlock() inline functions, both of which are defined in include/linux/spinlock.h:

void spin_unlock(spinlock_t *lock) (source)
void spin_lock(spinlock_t *lock) (source)

Now we will execute a demo code to demonstrate a dynamic spinlock kernel module on the Raspberry Pi.

Prerequisites for kernel module compilation

 $ sudo apt update
 $ sudo apt install raspberrypi-linux-headers

Sample code (filename: kernel_spinlock_example.c)

 #include <linux/module.h>
 #include <linux/kernel.h>
 #include <linux/init.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>

 struct dummy_struct {
     spinlock_t lock;
     unsigned int foo;
 };

 static struct dummy_struct *ds;

 static struct dummy_struct *demo_alloc_init_function(void) {
     struct dummy_struct *ds;
     ds = kmalloc(sizeof(struct dummy_struct), GFP_KERNEL);
     if (!ds)
         return NULL;
     spin_lock_init(&ds->lock);
     return ds;
 }

 static int __init my_module_init(void)
 {
     pr_info("Initializing Spinlock Example Module\n");
     ds = demo_alloc_init_function();
     if (!ds)
     {
         pr_alert("Allocation failed\n");
         return -ENOMEM;
     }

     // Lock the spinlock
     spin_lock(&ds->lock);
     pr_info("Spinlock acquired\n");

     // Simulate a critical section
     ds->foo = 42;
     pr_info("Critical section executed, foo = %u\n", ds->foo);

     // Unlock the spinlock
     spin_unlock(&ds->lock);
     pr_info("Spinlock released\n");

     return 0;
 }

 static void __exit my_module_exit(void)
 {
     pr_info("Exiting Spinlock Example Module\n");
     // Free the allocated memory
     if (ds)
     {
         kfree(ds);
         pr_info("Memory freed\n");
     }
 }

 module_init(my_module_init);
 module_exit(my_module_exit);

 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Suyog B <suyogburadkar@gmail.com>");
 MODULE_DESCRIPTION("A simple example kernel module using spinlocks on Raspberry Pi");
 MODULE_VERSION("1.0");

Makefile

 obj-m += kernel_spinlock_example.o

 all:
     make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

 clean:
     make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Its Time for kernel module building (Target Machine: Raspberry Pi Zero W 2)

 pi@SuyogB:~ $ uname -r
 6.1.21-v8+

 pi@SuyogB:~/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks $ ls
 kernel_spinlock_example.c  Makefile

 pi@SuyogB:~/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks $ make
 make -C /lib/modules/6.1.21-v8+/build M=/home/pi/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks modules
 make[1]: Entering directory '/usr/src/linux-headers-6.1.21-v8+'
   CC [M]  /home/pi/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks/kernel_spinlock_example.o
   MODPOST /home/pi/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks/Module.symvers
   CC [M]  /home/pi/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks/kernel_spinlock_example.mod.o
   LD [M]  /home/pi/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks/kernel_spinlock_example.ko
 make[1]: Leaving directory '/usr/src/linux-headers-6.1.21-v8+'

 pi@SuyogB:~/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks $ ls -ls
 total 40
 4 -rw-r--r-- 1 pi pi 1450 Jul 21 10:40 kernel_spinlock_example.c
 8 -rw-r--r-- 1 pi pi 7424 Jul 21 10:48 kernel_spinlock_example.ko
 4 -rw-r--r-- 1 pi pi  101 Jul 21 10:48 kernel_spinlock_example.mod
 4 -rw-r--r-- 1 pi pi  988 Jul 21 10:48 kernel_spinlock_example.mod.c
 4 -rw-r--r-- 1 pi pi 3584 Jul 21 10:48 kernel_spinlock_example.mod.o
 8 -rw-r--r-- 1 pi pi 4560 Jul 21 10:48 kernel_spinlock_example.o
 4 -rw-r--r-- 1 pi pi  173 Jul 21 10:42 Makefile
 4 -rw-r--r-- 1 pi pi  102 Jul 21 10:48 modules.order
 0 -rw-r--r-- 1 pi pi    0 Jul 21 10:48 Module.symvers

As we can see, our kernel module has been compiled successfully. The .ko file is present, indicating that the kernel build process was executed correctly. This process is called an out-of-tree kernel build.

Let's insert the .ko module and check the results:

 pi@SuyogB:~/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks $ sudo insmod kernel_spinlock_example.ko
 pi@SuyogB:~/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks $ dmesg | tail

Output:

 [   17.837522] Bluetooth: MGMT ver 1.22
 [   17.863066] NET: Registered PF_ALG protocol family
 [   39.810010] Bluetooth: RFCOMM TTY layer initialized
 [   39.810073] Bluetooth: RFCOMM socket layer initialized
 [   39.810124] Bluetooth: RFCOMM ver 1.11
 [  444.663733] kernel_spinlock_example: loading out-of-tree module taints kernel.
 [  444.664822] Initializing Spinlock Example Module
 [  444.664846] Spinlock acquired
 [  444.664860] Critical section executed, foo = 42
 [  444.664881] Spinlock released

The kernel module was successfully loaded, as indicated by the messages in the kernel log. The spinlock was acquired, the critical section executed (setting foo to 42), and the spinlock was released, confirming proper functionality.

Now let's unload the module:

 pi@SuyogB:~/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks $ sudo rmmod kernel_spinlock_example
 pi@SuyogB:~/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks $ dmesg | tail

Output:

 [   39.810010] Bluetooth: RFCOMM TTY layer initialized
 [   39.810073] Bluetooth: RFCOMM socket layer initialized
 [   39.810124] Bluetooth: RFCOMM ver 1.11
 [  444.663733] kernel_spinlock_example: loading out-of-tree module taints kernel.
 [  444.664822] Initializing Spinlock Example Module
 [  444.664846] Spinlock acquired
 [  444.664860] Critical section executed, foo = 42
 [  444.664881] Spinlock released
 [  473.583099] Exiting Spinlock Example Module
 [  473.583136] Memory freed

This output confirms that the module was successfully unloaded and the allocated memory was freed.

Cleaning the module

 pi@SuyogB:~/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks $ make clean
 make -C /lib/modules/6.1.21-v8+/build M=/home/pi/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks clean
 make[1]: Entering directory '/usr/src/linux-headers-6.1.21-v8+'
   CLEAN   /home/pi/linux-kernel-development-for-embedded-systems/Chapter_1_Spinlocks/Module.symvers
 make[1]: Leaving directory '/usr/src/linux-headers-6.1.21-v8+'

Github Repo (Chapter 1: Spinlocks): https://github.com/suyog44/linux-kernel-development-for-embedded-systems/tree/main/Chapter_1_Spinlocks

There are some limitations with spinlocks that we will discuss in the next chapter.

Chapter 1 : The kernel locks (Spinlock)

Spinlocks

Subscribe to my newsletter

Suyog Buradkar

Suyog Buradkar