Parallel Function Execution in Go Using Concurrency

Introduction

As part of my exploration of Golang, I came across a popular feature: first-class support for concurrency. I believe we all understand the benefit or importance of concurrency. In the HTTP way, when an endpoint needs to fetch data from multiple upstreams, aggregate the data and produce it as a response, Go concurrency helps to reduce the latency for that API request. Two features in Go, goroutines and channels make concurrency easier when used together.

Goroutines example: Run functions in parallel

Modern computers are equipped with processors, or CPUs, designed to efficiently handle multiple streams of code simultaneously. These processors are built with one or more "cores," each capable of running one code stream at a given time. To fully utilize the speed boost multiple cores offer, programs must be able to split into various streams of code. This division can be challenging, but Go was explicitly developed to simplify this process.

Go achieves this through a feature known as goroutines, special functions that can run alongside other goroutines. When a program is built to execute multiple streams of code simultaneously, it operates concurrently. Unlike traditional foreground operations, in which a function runs to completion before the following code executes, goroutines allow for background processing, enabling the following code to run while the goroutine is still active. This background operation ensures that the code doesn't block other processes from running.

Goroutines provide the advantage of running on separate processor cores simultaneously. For instance, if a computer has four processor cores and a program has four goroutines, all four can run concurrently. This simultaneous execution of multiple code streams on different cores is called parallel processing.

Jumping into the example, create a multifunc directory named go-concurrency-project.

mkdir go-concurrency-project
cd go-concurrency-project

Once you’re in the go-concurrency-project Directory, open a file named main.go using nano, or the editor of your choice:

nano main.go

Add the following code to the main.go file,

package main

import (
    "fmt"
)

func make(total int) {
    number := 0
    for number < total {
        number = number + 1
        fmt.Printf("Generated number %d\n", number)
    }
}

func print() {
    number := 0
    for number < 2 {
        number = number + 1
        fmt.Printf("Print: number %d\n", number)
    }
}

func main() {
    print()
    make(2)
}

Based on the above setup, make and print Functions are structured to run in sequence. make Accepts a number to generate up to and prints only five numbers.

This is how it will look like when we execute main.go,

go run make.go

// Output
Print: number 1
Print: number 2
Generated number 1
Generated number 2

If you notice, the function printed the output in sequence based on its execution pattern.

When running two functions synchronously, the program takes the total time for both functions to run. But if the functions are independent, you can speed up the program by running them concurrently using goroutines, potentially cutting the time in half. To run a function as a goroutine, use the go keyword before the function call. However, you need to add a way for the program to wait until both goroutines have finished running to ensure they all complete running.

To synchronize functions and wait for them to finish in Go, you can use a WaitGroup from the sync package. The WaitGroup primitive counts how many things it needs to wait for using the Add, Done, and Wait functions. The Add function increases the count, Done decreases the count, and Wait can be used to wait until the count reaches zero.

To do that update main.go,

package main

import (
    "fmt"
    "sync"
)

func make(total int, wg *sync.WaitGroup) {
    defer wg.Done()

    number := 0
    for number < total {
        number = number + 1
        fmt.Printf("Generated number %d", number)
    }
}

func print(wg *sync.WaitGroup) {
    defer wg.Done()

    number := 0
    for number < 2 {
        number = number + 1
        fmt.Printf("Print: number %d", number)
    }
}

func main() {
    var wg sync.WaitGroup

    wg.Add(2)
    go print(&wg)
    go make(2, &wg)

    fmt.Println("Awaiting....")
    wg.Wait()
    fmt.Println("Done!")
}

After declaring WaitGroup, specify how many processes to wait for. In the example, the goroutine waits for two Done calls before finishing. If not set before starting the goroutines, things might happen out of order, or the code may panic because wg doesn't know if it should wait for any Done calls.

Each function will use defer to call Done, which decreases the count by one after the function finishes. The main function is updated to include a call to Wait on the WaitGroup. This ensures that the main function waits until both functions call Done before continuing and exiting the program.

After saving your main.go execute the file,

go run main.go

// Output
Awaiting....
Generated number 1
Generated number 2
Print: number 1
Print: number 2
Done!

Your output may vary each time you run the program. With both functions running concurrently, the output depends on how much time Go and your operating system allocates to each function. Sometimes, each function runs entirely, and you'll see their complete sequences. Other times, the text will be interspersed.

Conclusion

If you’re interested in learning more about concurrency in Go, the Effective Go document created by the Go team provides much more detail. The Concurrency is not parallelism Go blog post is also an exciting follow-up about the relationship between concurrency and parallelism. These two terms are sometimes mistakenly thought to mean the same thing.

---

Thank you for reading this article! If you're interested in DevOps, Security, or Leadership for your startup, feel free to reach out at hi@iamkaustav.com or book a slot in my calendar. Don't forget to subscribe to my newsletter for more insights on my security and product development journey. Stay tuned for more posts!