WASM Microservices with Extism and Fiber
Philippe CharrièreTable of contents
Today, I will quickly show you how to serve Extism plugins (so Webassembly plugins) with the excellent framework Fiber. Fiber is a web framework for making HTTP servers with a similar spirit to Node.js frameworks like Express (which I have used many times in the past) or Fastify.
This article will be "slightly" longer than the previous ones because I also want to talk to you about my mistakes during my learning with Wasi.
Prerequisites
At best: to have read all the blog posts in this series: "Discovery of Extism (The Universal Plug-in System)"
At a minimum:
✋ Pay attention (Mon 7 Aug 2023)
The Extism Go SDK on top of Wazero. Now, the SDK is purely written in Go (no more dependency on cgo 🎉). However, the new SDK is not released yet. So to be able to use it and build your projects, you must:
clone the new repository:
git clonegit@github.com:extism/go-sdk.gitadd a local reference of this git repository into the
go.modfile of your projects:replacegithub.com/extism/extism=> ../go-sdk
https://twitter.com/mhmd_azeez/status/1688198055986622464
Creating an HTTP server as a host application
Start by creating a go.mod file with the command go mod init first-http-server, then a main.go file with the following content:
package main
import (
"context"
"fmt"
"net/http"
"os"
"github.com/extism/extism"
"github.com/gofiber/fiber/v2"
"github.com/tetratelabs/wazero"
)
func main() {
// Parameters of the program 0️⃣
wasmFilePath := os.Args[1:][0]
wasmFunctionName := os.Args[1:][1]
httpPort := os.Args[1:][2]
ctx := context.Background()
// Define the path to the wasm file 1️⃣
manifest := extism.Manifest{
Wasm: []extism.Wasm{
extism.WasmFile{
Path: wasmFilePath},
},
}
config := extism.PluginConfig{
ModuleConfig: wazero.NewModuleConfig().WithSysWalltime(),
EnableWasi: true,
}
// Load the wasm plugin 2️⃣
pluginInst, err := ctx.PluginFromManifest(manifest, []extism.Function{}, true)
if err != nil {
panic(err)
}
// Create an instance of Fiber application 3️⃣
app := fiber.New(fiber.Config{DisableStartupMessage: true})
// Create a route "/" and a handler to call the wasm function 4️⃣
app.Post("/", func(c *fiber.Ctx) error {
params := c.Body()
// Call the wasm function 5️⃣
// with a string parameter
_, out, err := pluginInst.Call(wasmFunctionName, params)
if err != nil {
fmt.Println(err)
c.Status(http.StatusConflict)
return c.SendString(err.Error())
} else {
// Send the HTTP response to the client 6️⃣
c.Status(http.StatusOK)
return c.SendString(string(out))
}
})
// Start the HTTP server 7️⃣
fmt.Println("🌍 http server is listening on:", httpPort)
app.Listen(":" + httpPort)
}
Some of the code is already familiar if you have read the previous articles.
0: use the program parameters to pass it the following information: the wasm plugin's path, the function's name to call and the HTTP port.
1: define a manifest with properties, including the Wasm file path.
2: load the Wasm plugin.
3: create a Fiber application.
4: create a route "/" that will be triggered by an HTTP request of type
POST.5: call the function of the plugin.
6: return the result (the HTTP response).
7: start the server.
Start the server and serve the WASM plugin
We will use the wasm plugin developed in Rust from our previous article: Create a Webassembly plugin with Extism and Rust.
Start the application as follows:
go run main.go \
path_to_the_plugin/hello.wasm \
hello \
8080
You should get this:
🌍 http server is listening on: 8080
And now, make an HTTP request:
curl -X POST \
http://localhost:8080 \
-H 'content-type: text/plain; charset=utf-8' \
-d '😄 Bob Morane'
echo ""
And you will get this:
{"message":"🦀 Hello 😄 Bob Morane"}
Stressing the application ... It's a disaster!
I always check the behaviour of my web services by "stressing" them with the utility Hey, which is extremely easy to use (especially with CI jobs, for example, to check the performance before and after changes).
So I'm going to stress my service for the first time with the following command:
hey -n 300 -c 1 -m POST \
-d 'John Doe' \
"http://localhost:8080"
So I will make 300 HTTP requests to my service with one single connection.
And I'm going to get a report like this (it's just an excerpt):
Summary:
Total: 0.0973 secs
Slowest: 0.0125 secs
Fastest: 0.0001 secs
Average: 0.0003 secs
Requests/sec: 3082.8745
Total data: 9900 bytes
Size/request: 33 bytes
Status code distribution:
[200] 300 responses
I work with a Mac M1 Max
Now, let's check the behaviour of the service with multiple connections at the same time:
hey -n 300 -c 100 -m POST \
-d 'John Doe' \
"http://localhost:8080"
So I will make 300 HTTP requests to my service with 100 simultaneous connections.
And this time, my HTTP server will crash! And in the load test report, you will see that most of the requests are in error (it's just an excerpt):
Status code distribution:
[200] 3 responses
Error distribution:
[3] Post "http://localhost:8080": EOF
[196] Post "http://localhost:8080": dial tcp 127.0.0.1:8080: connect: connection refused
[1] Post "http://localhost:8080": read tcp 127.0.0.1:38552->127.0.0.1:8080: read: connection reset by peer
[1] Post "http://localhost:8080": read tcp 127.0.0.1:38568->127.0.0.1:8080: read: connection reset by peer
But what happened?
In the paragraph "WASI" of the first article of the series, I explained that the way to exchange values other than numbers between the host application and the wasm plugin "guest" is to use the shared webassembly memory.
I encourage you to read this excellent article on the subject: A practical guide to WebAssembly memory by Radu Matei (CTO at FermyonTech).
You can also read this one, written by yours truly: WASI, Communication between Node.js and WASM modules with the WASM buffer memory
But let's get back to our problem. It is very simple: 100 connections were trying simultaneously to access this shared memory, so there was a "collision" because this memory is meant to be shared between the host application and only one "guest" at a time.
Therefore, we need to solve this problem to make our application usable.
Creating a second HTTP server, the "naive" solution
My first approach was to move the loading of the plugin from the manifest and its instantiation inside the HTTP handler to guarantee that for a given request, there will be only one access to the shared memory:
package main
import (
"context"
"fmt"
"net/http"
"os"
"github.com/extism/extism"
"github.com/gofiber/fiber/v2"
"github.com/tetratelabs/wazero"
)
func main() {
wasmFilePath := os.Args[1:][0]
wasmFunctionName := os.Args[1:][1]
httpPort := os.Args[1:][2]
ctx := context.Background()
config := extism.PluginConfig{
ModuleConfig: wazero.NewModuleConfig().WithSysWalltime(),
EnableWasi: true,
}
manifest := extism.Manifest{
Wasm: []extism.Wasm{
extism.WasmFile{
Path: wasmFilePath},
},
}
app := fiber.New(fiber.Config{DisableStartupMessage: true})
app.Post("/", func(c *fiber.Ctx) error {
params := c.Body()
// Load the wasm plugin 1️⃣
pluginInst, err := extism.NewPlugin(ctx, manifest, config, nil)
if err != nil {
fmt.Println(err)
c.Status(http.StatusConflict)
return c.SendString(err.Error())
}
// Call the wasm function 2️⃣
// with a string parameter
_, out, err := pluginInst.Call(wasmFunctionName, params)
if err != nil {
fmt.Println(err)
c.Status(http.StatusConflict)
return c.SendString(err.Error())
} else {
c.Status(http.StatusOK)
return c.SendString(string(out))
}
})
fmt.Println("🌍 http server is listening on:", httpPort)
app.Listen(":" + httpPort)
}
1: load the Wasm plugin.
2: call the function of the plugin.
So I launched my new HTTP server:
go run main.go \
path_to_the_plugin/hello.wasm \
hello \
8080
And I did some load tests:
hey -n 300 -c 100 -m POST \
-d 'John Doe' \
"http://localhost:8080"
And I got this report:
Summary:
Total: 7.6182 secs
Slowest: 4.6650 secs
Fastest: 0.0857 secs
Average: 2.0480 secs
Requests/sec: 39.3794
Status code distribution:
[200] 300 responses
So, it's excellent; everything works! 🎉 But the number of requests per second seems small. Less than 40 requests per second, compared to the 3000 requests per second of the first test; it's ridiculous 😞. But at least my application works.
But never hesitate to ask for help (that's why Open Source is a fabulous model).
the results of the tests can change, it depends of your computer, VM, etc.
Creating a third HTTP server, the "smart" solution
I was still annoyed by the poor performance of my MicroService. I had made a similar application with Node.js (remember the article: Writing Wasm MicroServices with Node.js and Extism), and the load tests gave me 1800 requests per second.
And with the Node.js version of the application, the wasm plugin was instantiated only once, and I had no memory collision problem 🤔.
This should have put me on track because Node.js applications use a "Single Threaded Event Loop Model", unlike Fiber which uses a "Multi-Threaded Request-Response" architecture to handle concurrent accesses. So that's why my Node.js application doesn't "crash".
It was Steve Manuel (CEO of Dylibso, but also the creator of Extism) who gave me the solution when I explained my problem to him during a discussion on Discord:
"So if you want thread-safety in Go HTTP handlers re-using plugins, you need to protect them with a mutex."
In fact, yes, it was so obvious (and also an opportunity to start studying what a mutex was).
So I followed Steve's advice, and I modified my code as follows:
package main
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"os"
"sync"
"github.com/extism/extism"
"github.com/gofiber/fiber/v2"
"github.com/tetratelabs/wazero"
)
// Store all your plugins in a normal Go hash map,
// protected by a Mutex 1️⃣
var m sync.Mutex
var plugins = make(map[string]extism.Plugin)
// Store the plugin 2️⃣
func StorePlugin(plugin extism.Plugin) {
plugins["code"] = plugin
}
// Retrieve the plugin 3️⃣
func GetPlugin() (extism.Plugin, error) {
if plugin, ok := plugins["code"]; ok {
return plugin, nil
} else {
return extism.Plugin{}, errors.New("🔴 no plugin")
}
}
func main() {
wasmFilePath := os.Args[1:][0]
wasmFunctionName := os.Args[1:][1]
httpPort := os.Args[1:][2]
ctx := context.Background()
config := extism.PluginConfig{
ModuleConfig: wazero.NewModuleConfig().WithSysWalltime(),
EnableWasi: true,
}
manifest := extism.Manifest{
Wasm: []extism.Wasm{
extism.WasmFile{
Path: wasmFilePath},
},
}
// Create an instance of the plugin 4️⃣
pluginInst, err := extism.NewPlugin(ctx, manifest, config, nil)
if err != nil {
log.Println("🔴 !!! Error when loading the plugin", err)
os.Exit(1)
}
// Save the plugin in the map 5️⃣
StorePlugin(plugin)
app := fiber.New(fiber.Config{DisableStartupMessage: true})
app.Post("/", func(c *fiber.Ctx) error {
params := c.Body()
// Lock the mutex 6️⃣
m.Lock()
defer m.Unlock()
// Get the plugin 7️⃣
pluginInst, err := GetPlugin()
if err != nil {
log.Println("🔴 !!! Error when getting the plugin", err)
c.Status(http.StatusInternalServerError)
return c.SendString(err.Error())
}
_, out, err := pluginInst.Call(wasmFunctionName, params)
if err != nil {
fmt.Println(err)
c.Status(http.StatusConflict)
return c.SendString(err.Error())
} else {
c.Status(http.StatusOK)
return c.SendString(string(out))
}
})
fmt.Println("🌍 http server is listening on:", httpPort)
app.Listen(":" + httpPort)
}
1: create a map protected by a mutex. This map will be used to "protect" the wasm plugin.
2: create a function to save the plugin in the map.
3: create a function to retrieve the plugin from the map.
4: create an instance of the wasm plugin.
5: save this instance in the map.
6: lock the mutex and use
deferto unlock it at the end of execution.7: get the plugin from the protected map.
Once this modification was done, I relaunched my HTTP server:
go run main.go \
path_to_the_plugin/hello.wasm \
hello \
8080
And I reran some load tests:
hey -n 300 -c 100 -m POST \
-d 'John Doe' \
"http://localhost:8080"
And I got this:
Summary:
Total: 0.0365 secs
Slowest: 0.0280 secs
Fastest: 0.0001 secs
Average: 0.0092 secs
Requests/sec: 8207.9604
Total data: 9900 bytes
Size/request: 33 bytes
Status code distribution:
[200] 300 responses
The new HTTP server version could handle up to more than 8000 requests per second! 🚀
Not bad, right? That's all for today. A huge thank you to Steve Manuel for his help. I learned a lot because I dared to ask for help. So, when you struggle with something and can't find a solution, don't hesitate to ask around.
See you soon for the next article. 👋
Subscribe to my newsletter
Read articles from Philippe Charrière directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by