Writing Host Functions in Go with Extism

In this article:

• Following the same principle as the previous article: Write a host function with the Extism Host SDK, we will modify the host application developed in Go from the article Run Extism WebAssembly plugins from a Go application to add a host function developed by us.

• We will use the same plugin as the one modified in the previous article in the section Modify the Wasm plugin to be able to call the host function.

Prerequisites

You will need:

• Go (v1.20) and TinyGo (v0.28.1) to compile the plugins

• Extism 0.4.0: Install Extism

• And at least you should have read the previous article: Write a host function with the Extism Host SDK (but probably all the articles in the series).

✋ Pay attention (Mon 7 Aug 2023)

The Extism Go SDK on top of Wazero. Now, the SDK is purely written in Go (no more dependency on cgo 🎉). However, the new SDK is not released yet. So to be able to use it and build your projects, you must:

• clone the new repository: git clone git@github.com:extism/go-sdk.git

• add a local reference of this git repository into the go.mod file of your projects: replace github.com/extism/extism => ../go-sdk

https://twitter.com/mhmd_azeez/status/1688198055986622464

Modifying the host application written in Go

The objective is the same as the previous article: to develop a host function that retrieves messages stored in the host application's memory based on a key. For this, we will use a Go Map. And this function will be used (called) by the Wasm plugin.

Important: To implement host functions, Extism's Go Host SDK uses the "Golang CGO" package (which allows invoking C code from Go and vice versa).

See the documentation: go-host-sdk/#host-functions

Here is the modified code of the application:

package main

import (
    "context"
    "fmt"

    "github.com/extism/extism"
    "github.com/tetratelabs/wazero" // 1️⃣
    "github.com/tetratelabs/wazero/api" // 2️⃣
)


// 3️⃣ define a map with some records
var memoryMap = map[string]string{
    "hello": "👋 Hello World 🌍",
    "message": "I 💜 Extism 😍",
}

func main() {

    ctx := context.Background()

    // use the updated plugin
    path := "../12-simple-go-mem-plugin/simple.wasm"

    config := extism.PluginConfig{
        ModuleConfig: wazero.NewModuleConfig().WithSysWalltime(),
        EnableWasi:   true,
    }

    manifest := extism.Manifest{
        Wasm: []extism.Wasm{
            extism.WasmFile{
                Path: path},
        }}

    // 4️⃣ host function definition 
    // (callable by the Wasm plugin)
    memory_get := extism.HostFunction{
        Name:      "hostMemoryGet",
        Namespace: "env",
        Callback: func(ctx context.Context, plugin *extism.CurrentPlugin, userData interface{}, stack []uint64) {
            // 5️⃣ read the value of the function arguments
            // the arguments are passed to the function
            // when it be called by the wasm plugin
            offset := stack[0]
            bufferInput, err := plugin.ReadBytes(offset)

            if err != nil {
                fmt.Println("🥵", err.Error())
                panic(err)
            }

            keyStr := string(bufferInput)
            fmt.Println("🟢 keyStr:", keyStr)

            // 6️⃣ get the associated string value
            returnValue := memoryMap[keyStr]

            // 7️⃣ copy the return value to the memory
            plugin.Free(offset)
            offset, err = plugin.WriteBytes([]byte(returnValue))
            if err != nil {
                fmt.Println("😡", err.Error())
                panic(err)
            }

            stack[0] = offset
        },
        Params:  []api.ValueType{api.ValueTypeI64},
        Results: []api.ValueType{api.ValueTypeI64},
    }

    // 8️⃣ define a slice of host functions
    hostFunctions := []extism.HostFunction{
        memory_get,
    }

    // 9️⃣
    pluginInst, err := extism.NewPlugin(
        ctx, 
        manifest, 
        config, 
        hostFunctions)

    if err != nil {
        panic(err)
    }

    _, res, err := pluginInst.Call(
        "say_hello",
        []byte("👋 Hello from the Go Host app 🤗"),
    )

    if err != nil {
        fmt.Println("😡", err)
        //os.Exit(1)
    } else {
        //fmt.Println("🙂", res)
        fmt.Println("🙂", string(res))
    }
}

• 1: import the wazero package (Wazero is a WebAssembly runtime)

• 2: import the wazero/api package

• 3: Create a map with some elements. The host function will use this.

• 4: Definition of the host function memory_get. Do not forget to export the function with //export memory_get (a host function will always have the same signature).

• 5: When the Wasm plugin calls the host function, the passing of parameters is done using the memory shared between the plugin and the host. currentPlugin.InputString(unsafe.Pointer(&inputSlice[0])) is used to fetch this information from shared memory. keyStr is a string that contains the key to retrieve a value from the map.

• 6: Fetch the value associated with the key in the map.

• 7: Copy the obtained value into memory to allow the Wasm plugin to read it.

• 8: Define an array of host functions. In our case, we create only one, where "hostMemoryGet" will be the alias of the function "seen" by the Wasm plugin, []extism.ValType{extism.I64} represents the input parameter type and the return parameter type (remember that Wasm functions only accept numbers - and in our case, these numbers contain the positions and sizes of values in shared memory), and finally, C.memory_get which is the definition of our host function.

• 9: Create an instance of the Wasm plugin by passing the array of host functions as a parameter.

Reminder: The code of the modified Wasm plugin (written in Go) is here: Plugin Wasm Go

Running the Application

To test your new host application, run the following command:

go run main.go

And you will get the following output, with the messages from each of the keys in the Go map:

🙂 👋 Hello 👋 Hello from the Go Host app 🤗
key: hello, value: 👋 Hello World 🌍
key: message, value: I 💜 Extism 😍

🎉 There you have it; we have written a host function in Go, usable with the same Wasm plugin (without modifying it). So this plugin can call a host function written in JavaScript, Go, Rust, ... as long as the application has implemented this host function with the same signature.