Wazero Cookbook - Part one: WASM function & Host application

The WASI Land of Tools is growing fastly. So I tried to map the various territories to help you find your way around:

You can develop "from scratch" WASM modules and host applications with the WASM runtimes SDK.
You can develop WASM modules and host applications with toolkit frameworks (this will be the subject of another blog post series).
You can develop WASM modules and use ready-to-use applications (like HTTP servers serving WASM functions) with applicative frameworks (this will be the subject of another blog post series).

Today, we will focus on developing "from scratch" WASM modules and host applications with a WASM runtimes SDK. This new blog post series is about Wazero, the only zero-dependency WebAssembly runtime written in Go.

Requirements

You need to install the following:

GoLang and TinyGo (previous blog post: "Install Go and TinyGo").

And it's not mandatory, but it can help if you start from zero. You can read these former blog posts:

Take some coffee, and let's get started!

Chapter 1: Call a function (with numbers)

Calling a WASM function from GoLang with Wazero is simple. I will use the TinyGo compiler to build the WASM module and the GoLang compiler (and the Wazero SDK) to build the host application that calls the WASM function.

Write a WASM function

We will first write a WASM module with a add function that takes 2 numbers and returns the sum. Create a new GoLang module and a add.go file (in a function directory):

go mod init function/add
touch add.go

The go.mod file should look like this:

module function/add

go 1.20

This is the content of the add.go file:

package main

func main() {}

//export add
func add(a int, b int) int {
    return a + b
}

👋 Important: to make the add function callable from the host, we need to add the export add comment above the function.

Build and run the function

Build the WASM file with the below command:

tinygo build -o add.wasm -scheduler=none --no-debug -target wasi ./add.go

You can test the function with the following commands using the wasmer CLI or the wasmedge CLI:

wasmedge --reactor add.wasm add 18 24
wasmer add.wasm --invoke add 12 30 # 42

Call the `add` function from GoLang

Loading and instantiating a WASM module and calling the function from GoLang with Wazero is pretty simple. Create a new GoLang module and a main.go file (in a host directory):

go mod init host
touch main.go

Add the Wazero dependency to the go.mod file:

module host

go 1.20

require github.com/tetratelabs/wazero v1.0.1

This is the content of the main.go file:

package main

import (
    "context"
    "fmt"
    "github.com/tetratelabs/wazero"
    "github.com/tetratelabs/wazero/imports/wasi_snapshot_preview1"
    "log"
    "os"
)

func main() {
    // Choose the context to use for function calls.
    ctx := context.Background()

    // Create a new WebAssembly Runtime.
    runtime := wazero.NewRuntime(ctx)

    // This closes everything this Runtime created.
    defer runtime.Close(ctx) 

    // Instantiate WASI
    wasi_snapshot_preview1.MustInstantiate(ctx, runtime)

    // Load the WebAssembly module
    wasmPath := "../function/add.wasm"
    addWasm, err := os.ReadFile(wasmPath)

    if err != nil {
        log.Panicln(err)
    }

    // Instantiate the guest Wasm into the same runtime. 
    // It exports the `add` function, 
    // implemented in WebAssembly.
    mod, err := runtime.Instantiate(ctx, addWasm)
    if err != nil {
        log.Panicln(err)
    }

    // Get the reference to the WebAssembly function: "add"
    addFunction := mod.ExportedFunction("add")

    // Now, we can call "add"
    // result []uint64
    result, err := addFunction.Call(ctx, 20, 22)
    if err != nil {
        log.Panicln(err)
    }

    fmt.Println("result:", result[0])    
}

Build and run the host application

Build the host application with the below command:

go build

Run the application

./host
# you can use: `go run main.go`

You should get this output:

result: 42

Chapter 2: Call a function (with strings)

Now, I would like to write some more complex functions. My main requirements are:

Use a string as a parameter of a function
Return a string from the function

In this blog post: WASI, Communication between Node.js and WASM modules with the WASM buffer memory, we saw that using strings with WebAssembly is not always trivial. We will use the same recipes as before but with Wazero. So, we'll need some "helpers" for that.

Remember this: a "callable" (by the host) function of a WASM module can only take numbers as parameters and can only return a number. But, the WASM module and the host application share a memory buffer, and it's used to pass data between the host and the module. So, when you are not dealing with only numbers, and for example, you need to use strings:

If the host needs to pass a value to the wasm function, it will copy the string to the WASM memory buffer
Then, the "wasm function" will read the string from the memory buffer
And the "wasm function" will write the returned value to the WASM memory buffer
Then, the "wasm function" will return the memory position and the size of the value into only one value
And finally, the host will decode the position and the size and then read the returned value from the memory

So, let's do it!

Create the new WASM module

I would like to be able to call a wasm function (from the host application) with a string as a parameter, and then this function will return a string message to the host application.

Create a new GoLang module and a hello.go file (in a function directory):

go mod init function/hello
touch hello.go

Add this content to the hello.go file:

package main

import "unsafe"

func main () {}

//export hello
func hello(valuePosition *uint32, length uint32) uint64 {

}

Read the WASM memory buffer

The function needs to read the memory to get the value "sent" by the host application. Add this function to your module:

// readBufferFromMemory returns a buffer from WebAssembly
func readBufferFromMemory(bufferPosition *uint32, length uint32) []byte {
    subjectBuffer := make([]byte, length)
    pointer := uintptr(unsafe.Pointer(bufferPosition))
    for i := 0; i < int(length); i++ {
        s := *(*int32)(unsafe.Pointer(pointer + uintptr(i)))
        subjectBuffer[i] = byte(s)
    }
    return subjectBuffer
}

👋 Important: the readBufferFromMemory extract an array of bytes from the WASM buffer memory. This array contains the value "sent" by the host application.

Copy data to the WASM memory buffer

The function needs to copy the data to the WASM memory buffer to "send" the new value to the host application. Add this function to your module:

// copyBufferToMemory returns a single value (a kind of pair with position and length)
func copyBufferToMemory(buffer []byte) uint64 {
    bufferPtr := &buffer[0]
    unsafePtr := uintptr(unsafe.Pointer(bufferPtr))

    ptr := uint32(unsafePtr)
    size := uint32(len(buffer))

    return (uint64(ptr) << uint64(32)) | uint64(size)
}

👋 Important: the copyBufferToMemory returns the position and the size of the buffer in memory into only one value using the bit-shifting method.

Finalize the `hello` function

Now, complete the hello function like this:

//export hello
func hello(valuePosition *uint32, length uint32) uint64 {

    // read the memory to get the parameter
    valueBytes := readBufferFromMemory(valuePosition, length)

    message := "Hello " + string(valueBytes)

    // call the handle function
    posSizePairValue := copyBufferToMemory([]byte(message))

    // return the value
    return posSizePairValue
}

This is the entire source code:

package main

import "unsafe"

func main () {}

//export hello
func hello(valuePosition *uint32, length uint32) uint64 {

    // read the memory to get the parameter
    valueBytes := readBufferFromMemory(valuePosition, length)

    message := "Hello " + string(valueBytes)

    // copy the value to memory
    posSizePairValue := copyBufferToMemory([]byte(message))

    // return the position and size
    return posSizePairValue
}

// readBufferFromMemory returns a buffer from WebAssembly
func readBufferFromMemory(bufferPosition *uint32, length uint32) []byte {
    subjectBuffer := make([]byte, length)
    pointer := uintptr(unsafe.Pointer(bufferPosition))
    for i := 0; i < int(length); i++ {
        s := *(*int32)(unsafe.Pointer(pointer + uintptr(i)))
        subjectBuffer[i] = byte(s)
    }
    return subjectBuffer
}

// copyBufferToMemory returns a single value (a kind of pair with position and length)
func copyBufferToMemory(buffer []byte) uint64 {
    bufferPtr := &buffer[0]
    unsafePtr := uintptr(unsafe.Pointer(bufferPtr))

    ptr := uint32(unsafePtr)
    size := uint32(len(buffer))

    return (uint64(ptr) << uint64(32)) | uint64(size)
}

Now, build the wasm module:

tinygo build -o hello.wasm -scheduler=none --no-debug -target wasi ./hello.go

Now let's move on to the host application.

Create the host application

We will create a new GoLang application. This application will use the Wazero SDK to call the hello WASM function. Create a new GoLang module and a main.go file (in a host directory):

go mod init host
touch main.go

Add the Wazero dependency to the go.mod file:

module host

go 1.20

require github.com/tetratelabs/wazero v1.0.1

This is the content of the main.go file:

// Package main of the host application
package main

import (
    "context"
    "fmt"
    "github.com/tetratelabs/wazero"
    "github.com/tetratelabs/wazero/imports/wasi_snapshot_preview1"
    "log"
    "os"
)

func main() {
    // Choose the context to use for function calls.
    ctx := context.Background()

    // Create a new WebAssembly Runtime.
    runtime := wazero.NewRuntime(ctx)

    // This closes everything this Runtime created.
    defer runtime.Close(ctx) 

    // Instantiate WASI
    wasi_snapshot_preview1.MustInstantiate(ctx, runtime)

    // Load the WebAssembly module
    wasmPath := "../function/hello.wasm"
    helloWasm, err := os.ReadFile(wasmPath)

    if err != nil {
        log.Panicln(err)
    }

    // Instantiate the guest Wasm into the same runtime. 
    // It exports the `hello` function, 
    // implemented in WebAssembly.
    mod, err := runtime.Instantiate(ctx, helloWasm)
    if err != nil {
        log.Panicln(err)
    }

    // Get the reference to the WebAssembly function: "hello"
    helloFunction := mod.ExportedFunction("hello")

    // Function parameter 1️⃣
    name := "Bob Morane"
    nameSize := uint64(len(name))

    // These function are exported by TinyGo 2️⃣
    malloc := mod.ExportedFunction("malloc")
    free := mod.ExportedFunction("free")

    // Allocate Memory for "Bob Morane" 3️⃣
    results, err := malloc.Call(ctx, nameSize)
    if err != nil {
        log.Panicln(err)
    }
    namePosition := results[0]

    // This pointer is managed by TinyGo, 
    // but TinyGo is unaware of external usage.
    // So, we have to free it when finished
    defer free.Call(ctx, namePosition)

    // Copy "Bob Morane" to memory 4️⃣
    if !mod.Memory().Write(uint32(namePosition), []byte(name)) {
        log.Panicf("out of range of memory size")
    }

    // Now, we can call "hello" 5️⃣
    // with the position and the size of "Bob Morane"
    // the result type is []uint64
    result, err := helloFunction.Call(ctx, namePosition, nameSize)
    if err != nil {
        log.Panicln(err)
    }

    // Extract the position and size of the returned value 6️⃣
    valuePosition := uint32(result[0] >> 32)
    valueSize := uint32(result[0])

    // Read the value from the memory 7️⃣
    if bytes, ok := mod.Memory().Read(valuePosition, valueSize); !ok {
        log.Panicf("out of range of memory size")
    } else {
        fmt.Println("Returned value :", string(bytes))
    }
}

- 1️⃣ `name := "Bob Morane"` is the value we want to send to the WASM function

- 2️⃣ `malloc` and `free` are function exported by TinyGo
- 3️⃣ We use `malloc` to allocate a place in memory (for the **"Bob Morane"** string).

Then, with the return value of `malloc`, 
we obtain a position in memory to copy the string:
`namePosition := results[0]`

- 4️⃣ We copy the string to the memory at the appropriate position

- 5️⃣ We call the function (using the position and the size of "Bob Morane" as parameters)
- 6️⃣ We extract the position and size of the returned value

- 7️⃣ 🎉 we can read the value from the memory

Now, build the host application:

go build

Run the application

./host
# you can use: `go run main.go`

You should get this output:

Returned value : Hello Bob Morane

👏 And that's it! You know how to develop a host application and WASM plugin for this application.

A quick recap

Copy "Bob Morane" to the WASM buffer memory
Call the WASM function
Read the value from the WASM buffer memory
Copy the returned value to the WASM buffer memory
Return the position and size of the returned value (into only one value)
Read the returned value "Hello Bob Morane" from the WASM buffer memory

🎉 Thanks for reading this first part of Wazero recipes! I hope you enjoyed it. In the next blog post, we'll see how to implement host functions with Wazero.

All examples are available here: https://github.com/wasm-university/wazero-step-by-step

Wazero Cookbook - Part One: WASM function & Host application

Table of contents

Requirements

Chapter 1: Call a function (with numbers)

Write a WASM function

Build and run the function

Call the `add` function from GoLang

Build and run the host application

Chapter 2: Call a function (with strings)

Create the new WASM module

Read the WASM memory buffer

Copy data to the WASM memory buffer

Finalize the `hello` function

Create the host application

A quick recap

Subscribe to my newsletter

Philippe Charrière

Philippe Charrière

Wazero Cookbook - Part One: WASM function & Host application

Table of contents

Requirements

Chapter 1: Call a function (with numbers)

Write a WASM function

Build and run the function

Call the add function from GoLang

Build and run the host application

Chapter 2: Call a function (with strings)

Create the new WASM module

Read the WASM memory buffer

Copy data to the WASM memory buffer

Finalize the hello function

Create the host application

A quick recap

Subscribe to my newsletter

Philippe Charrière

Philippe Charrière

Call the `add` function from GoLang

Finalize the `hello` function