"Garden Telemetry with Elixir: Building a Smart Monitoring System"

Inspired by and built upon Chapters 3 and 4 of "The Little Elixir & OTP Guidebook" by Benjamin Tan Wei Hao.

The code for this project can be found here.

Introduction

Let's build a feature that helps track and monitor the various aspects of a large garden and thought the weather API example in Benjamin's book was a great place to start for building a simple weather widget. It will consist of three parts:

Part I below will introduce the concept of a GenServer and walk through a synchronous call/3 and handle_call/3 function/callback implementation.
Part II will expand upon Part I and introduce the asynchronous cast/2 function and both handle_cast/2 and handle_info/2 callbacks.
Part III will integrate this feature into a LiveView application.

Getting Started

Let's start with our complete GenServer model as it is presented in Chapter 4 of "The Little Elixir & OTP Guidebook", though some of the naming conventions have been changed to expand on the book's example and integrate it into the domain of our application:

defmodule Garden.WeatherWorker do

  use GenServer # -> defines the callbacks required for GenServer


  # Client API - start_link/3, call/3, cast/2

  def start_link(_opts \\[]) do
    GenServer.start_link(__MODULE__, :ok, name: :garden_weather_worker)
  end 

  @spec get_temperature(atom() | pid() | {atom(), any()} | {:via, atom(), any()}, any()) :: any()
  def get_temperature(pid, location) do
    # wraps a call to GenServer.call/3 - passes in the pid, a tuple tagged `:location`
    # this is a synchronous call, so a reply from the server is expected
    GenServer.call(pid, {:location, location})
  end

  def get_stats(pid) do
    GenServer.call(pid, :get_stats)
  end

  @spec reset_stats(atom() | pid() | {atom(), any()} | {:via, atom(), any()}) :: :ok
  def reset_stats(pid) do
    GenServer.cast(pid, :reset_stats)
  end

  def stop(pid) do
    GenServer.cast(pid, :stop)
  end

  def terminate(reason, stats) do
    # we could write to a file, database, etc
    IO.puts "server terminated because of #{inspect reason}"
      inspect stats
    :ok
  end

  # Server API - init/1, handle_call/3, handle_cast/2

    # triggered by start_link/3
  def init(:ok) do
    # {:ok, state}
    {:ok, %{}}
  end


  # temperature_of/1 makes a request to the third-party API to get the location's temperature
  # if it succeeds, update_stats/2 is invoked to return a new map with the updated frequency of location.
  # then returns a three-element tuple, same as any other handle_call/3 is expected to return.
  def handle_call({:location, location}, _from, stats) do
    case temperature_of(location) do
      {:ok, temp} ->
        new_stats = update_stats(stats, location)
        {:reply, "#{temp}°F", new_stats}

        _ ->
          {:reply, :error, stats}
    end
  end

  def handle_call(:get_stats, _from, stats) do
    {:reply, stats, stats}
  end

  def handle_cast(:reset_stats, _stats) do
    {:noreply, %{}}
  end

  def handle_cast(:stop, stats) do
    {:stop, :normal, :ok, stats}
  end

  def handle_info(msg, stats) do
    IO.puts "received #{inspect(msg)}"
    {:noreply, stats}
  end



  # Helper Functions

  # a wrapper around the `url_for/1` function, which returns a URL in an ASCII-valid format.
    # URL is piped into an HTTPoison.get/3 function, issuing a GET request for the given URL.
    # result of the HTTPoison.get/3 call is piped into the parse_response/2 function.
  def temperature_of(location) do
    url_for(location) |> HTTPoison.get |> parse_response
  end

  # private helper function which returns a properly formatted URL (ASCII characters).
    # URI.encode/2 percent-codes all characters that require escaping.
  defp url_for(location) do
    location = URI.encode(location)
    "http://api.openweathermap.org/data/2.5/weather?q=#{location}&appid=#{apikey()}"
  end

  # last step in the function pipeline located in the body of the temperature_of/1 function).
    # the result of the HTTPoison.get/3 call in that pipeline returns an {:ok, %HTTPoison.Response{body: body, status_code: 200}} tuple.
    # the value of the :body key in the %HTTPoison.Response struct is set to the `body` variable in the function head's argument, via pattern matching.
    # the value assigned to the `body` variable via that process is piped into Jason.decode!/2 and then compute_temperature/1.
    # Jason.decode!/2 parses the values from JSON and converts into an Elixir map.
  defp parse_response({:ok, %HTTPoison.Response{body: body, status_code: 200}}) do
    body |> Jason.decode! |> compute_temperature
  end

  # second parse_response/1 function for error handling via pattern matching on function heads.
  defp parse_response(_) do
    :error
  end

  # helper function to convert unit from Kelvin (OpenWeatherMap default) to Celsius
  defp compute_temperature(json) do
    try do
      temp = (json["main"]["temp"] * 9/5 - 459.67) |> Float.round(1)
      {:ok, temp}
    rescue
      _ -> :error
    end
  end

  defp apikey do
    "b694cf0c2a42bb0015c7cea79d3c86e1"
  end


  defp update_stats(old_stats, location) do
    Map.update(old_stats, location, 1, &(&1 + 1))
  end


end

This is not the full implementation of our weather feature but it does what we need it to for now - sends a GET request to the OpenWeather API in response we retrieve a temperature value for the provided city.

What is a GenServer?

GenServers are an abstraction built on top of standard Erlang processes, designed to simplify and streamline the client-server interaction within our application. Erlang processes, and therefore GenServers, communicate via message passing. Each process has a mailbox and handles requests as they come in.

To model this client-server interaction, GenServer functions and their respective callbacks are split into a Client API and a Server API. This provides a standard way of working across GenServers and within a Supervision tree. Let's break it down.

start_link/3

This is a plain Elixir function called start_link/1. Within the body is a call to GenServer.start_link/3.

 def start_link(_opts \\ []) do
    GenServer.start_link(__MODULE__, :ok, name: :garden_weather_worker)
  end

If you're new to Elixir, take notice of the distinction between the two start_link functions. GenServer.start_link/3 is a function defined in the GenServer module and its job is to start the GenServer's process.

We wrap that function call in our own start_link/1 function, which we've defined in our application's Garden.WeatherWorker module:

defmodule Garden.WeatherWorker do

  use GenServer # -> defines the callbacks required for GenServer


  def start_link(_opts \\[]) do
    GenServer.start_link(__MODULE__, :ok, name: 
    :garden_weather_worker)
  end 

  # ......
  # ......

end

This increases flexibility, as we can modify the code before the actual GenServer.start_link/3 call is made by adding it to the function body if needed, while keeping the language of the API clean and consistent. Also, if your GenServer is part of a supervision tree, the Supervisor will call this function to start its process.

init/1

Calling this start_link/3 function triggers the corresponding callback function, init/1:

  def init(:ok) do
    # {:ok, state}
    {:ok, %{}}
  end

This is our first callback. We've started our GenServer process with GenServer.start_link/3. Let's take note of the second argument of that function - init_arg. The value for our init_arg argument is the atom :ok, which passed to the init/1 callback function via start_link/3. It will then pattern match on the argument in its function head.

note: :ok is the common convention for this argument when initializing a GenServer process, especially when no additional data is specified.

Now that our process has been initialized, let's further explore the client-server interaction that GenServer helps to unify and streamline for us - and for any developer that might have to reason about our code in the future.

Sync vs. Async

Naturally, not every request will serve the same purpose. Just as in real life, the requisite or optimized mode of communication is context dependent. To mimic this we use synchronous and asynchronous message passing between processes.

synchronous calls require a reply, while asynchronous calls do not.

Because asynchronous calls are not contingent on a direct response to complete the request, they are often referred to as fire and forget.

So how are synchronous and asynchronous messages handled? Via the call/3 and cast/2 and their respective callback functions, handle_call/3 and handle_cast/2.

call/3 & handle_call/3

When call/3 is executed, a request is sent to the server. Let's take a look at our call/3 function, placed within the body of get_temperature/2:

 def get_temperature(pid, location) do
    # wraps a call to GenServer.call/3 - passes in the pid, a tuple tagged `:location
    # this is a synchronous call, so a reply from the server is expected
    GenServer.call(pid, {:location, location})
  end

And our corresponding handle_call/3 callback:

 def handle_call({:location, location}, _from, stats) do
    case temperature_of(location) do
      {:ok, temp} ->
        new_stats = update_stats(stats, location)
        {:reply, "#{temp}°F", new_stats}

        _ ->
          {:reply, :error, stats}
    end

The body of our handle_call/3 function uses a case statement to pattern match on the incoming request sent to this function via the Genserver.call/3 function in our Client API:

  case temperature_of(location) do
      {:ok, temp} ->
        new_stats = update_stats(stats, location)
        {:reply, "#{temp}°F", new_stats}

        _ ->
          {:reply, :error, stats}
    end

What's important here is that our case statement uses the result of calling the temperature_of/1 helper function as the value to be matched on. A successful pattern match results in a successful call to that function, which returns an {:ok, temp} tuple. :ok is the first element of that tuple, so it matches. What is bound to temp is the actual temperature fetched via the OpenWeather API call.

See the other pattern match here? Think about where the temperature_of/1 function is actually receiving its location from -it comes packaged in the second argument of the get_temperature/2 function, which is the wrapper containing Genserver.call/3.

The result is bound to location, which is the second element of the {:location, location} tuple. The handle_call/3 also contains this {:location, location} tuple. This is essentially the address system for this type of Genserver Client/Server communication: When these tuples match, callback is successfully executed.

note: This also allows for the ability to have several handle_call or handle_cast callbacks without having to uniquely name them or encase their logic in unique wrapper functions and call them specifically - they will simply pattern match on the correct callback, not unlike the way a case statement executes the first clause that it matches on.

All of this successful matching will trigger the first clause of our case statement:

{:ok, temp} ->
new_stats = update_stats(stats, location)
     {:reply, "#{temp}°F", new_stats}

We're calling one of our other helper functions, update_stats/2, and assigning the result of that call to the new_stats variable. This is how we always return the most up-to-date state when calling get_stats/1 to view the list of every city or town we've requested a temperature for, along with how many times each location was called. More detail on that in Part II. For now, that's enough to know what's happening here.

We now have our complete return tuple, {:reply, "#{temp}", new_stats}. Now, when we call our get_temperature/2 function (which remember is really just our call/3 function under the hood) we will trigger the handle_call/3 callback that matches, and return its response back to the Client portion of the Genserver.

Since call/3 is a synchronous call, it requires :reply as the first element in the return tuple. As you could guess, since cast/2 is asynchronous and is not concerned with a response, :noreply is used.

Makes sense that if we have a tuple with :ok as the first value set as the return of a successful function execution, then we must also account for an unsuccessful call, so we have our _ operator present as a catch-all clause for any result that does not successfully pattern match on the first clause.

note: Always include this catch all as the last clause in any case statement. When the _ operator is placed anywhere else it will always match and prematurely complete the case statement without allowing any clause that follows it the potential to match.

That was a lot. Let's see it in action:

Garden.WeatherWorker
iex(2)> {:ok, pid} = WeatherWorker.start_link()
{:ok, #PID<0.342.0>}
iex(3)> WeatherWorker.get_temperature(pid, "Tallahassee")
"85°F"

Don't forget to refer to the complete GenServer module at the beginning of the article for reference - Part II coming soon!

Garden Telemetry : Weather API

Table of contents