Building a Golang Syslog Server: A Journey Through the Digital Cosmos

Jaakko LeskinenJaakko Leskinen
6 min read

Introduction

Welcome, fellow traveler, to the whimsical world of system logging—a realm both mundane and essential. In this guide, we embark on an adventure to craft a Golang-based syslog server, inspired by the delightful absurdity of Douglas Adams’ universe. Much like the planet Earth in Mostly Harmless, our server will be mostly harmless—and delightfully efficient. So, grab your towel and dive into the cosmos of code.

The full code is available on my GitHub repository: jleski/wetherly.

Laying the Groundwork

Before launching our metaphorical spaceship, let’s prepare the essentials:

  • Go: Our language of choice, as sleek and reliable as a well-tuned spaceship.

  • Docker: Ensuring our server runs smoothly across the galaxy of environments.

  • Task: A task runner to automate the myriad tasks needed for a shipshape server.

  • Helm: For deploying in the Kubernetes nebula with precision.

  • Netcat (nc): The Swiss Army knife of networking, for sending test messages.

  • Golangci-lint: Optional but invaluable for polished code.

Setting Up the Environment

Clone the repository and prepare your development environment:

git clone https://github.com/jleski/wetherly.git
cd wetherly
task dev:setup

This installs the necessary dependencies and readies your workspace.

Building the Server

Our syslog server, much like a Vogon constructor fleet, is a marvel of precision. Its main components reside in main.go, handling syslog messages per the RFC5424 standard.

Key Components

  • Listener: Listening on port 6601 for intergalactic messages.

  • Parser: Using the github.com/influxdata/go-syslog/v3 library to decode messages.

  • Handler: Spawning a goroutine for each connection, ensuring concurrency.

Here’s a sneak peek of the main function:

func main() {
    printStartupInfo()

    listener, err := net.Listen("tcp", SYSLOG_PORT)
    if err != nil {
        log.Fatalf("%sError creating TCP listener: %v%s", RedColor, err, ResetColor)
    }
    defer listener.Close()

    fmt.Printf("%s✅ Server is ready to accept connections%s\n\n", GreenColor, ResetColor)

    for {
        conn, err := listener.Accept()
        if err != nil {
            log.Printf("%sError accepting connection: %v%s", RedColor, err, ResetColor)
            continue
        }

        fmt.Printf("%s📥 New connection from %s%s\n", GreenColor, conn.RemoteAddr(), ResetColor)
        go handleConnection(conn)
    }
}

Main Functions of the Syslog Server

1. main()

Sets up the server to listen for connections and processes them concurrently.

func main() {
    printStartupInfo()

    listener, err := net.Listen("tcp", SYSLOG_PORT)
    if err != nil {
        log.Fatalf("%sError creating TCP listener: %v%s", RedColor, err, ResetColor)
    }
    defer listener.Close()

    fmt.Printf("%s✅ Server is ready to accept connections%s\n\n", GreenColor, ResetColor)

    for {
        conn, err := listener.Accept()
        if err != nil {
            log.Printf("%sError accepting connection: %v%s", RedColor, err, ResetColor)
            continue
        }

        fmt.Printf("%s📥 New connection from %s%s\n", GreenColor, conn.RemoteAddr(), ResetColor)
        go handleConnection(conn)
    }
}

2. printStartupInfo()

Prints a colorful startup banner and server details.

func printStartupInfo() {
    fmt.Print(CyanColor)
    fmt.Print(BANNER)
    fmt.Print(ResetColor)

    hostname, err := os.Hostname()
    if err != nil {
        hostname = "unknown"
    }

    fmt.Print(GreenColor)
    fmt.Printf("🚀 Starting Wetherly Syslog Server...\n")
    fmt.Printf("📅 Time: %s\n", time.Now().Format(time.RFC1123))
    fmt.Printf("💻 Hostname: %s\n", hostname)
    fmt.Printf("🔌 Protocol: TCP\n")
    fmt.Printf("🎯 Port: 6601\n")
    fmt.Printf("📦 Buffer Size: %d bytes\n", BUFFER_SIZE)
    fmt.Print(ResetColor)
    fmt.Println("==========================================")
}

3. handleConnection(conn net.Conn)

Processes each connection, reading and parsing messages.

func handleConnection(conn net.Conn) {
    defer conn.Close()

    buffer := make([]byte, BUFFER_SIZE)
    for {
        n, err := conn.Read(buffer)
        if err != nil {
            if err.Error() != "EOF" {
                log.Printf("%sError reading from connection: %v%s", RedColor, err, ResetColor)
            }
            fmt.Printf("%s📤 Connection closed from %s%s\n", YellowColor, conn.RemoteAddr(), ResetColor)
            return
        }

        message := string(buffer[:n])
        timestamp := time.Now().Format("2006-01-02 15:04:05")

        if strings.HasPrefix(message, "<") {
            parsedMsg, err := parseRFC5424Message(message)
            if err != nil {
                fmt.Printf("%sError parsing RFC5424 message: %v%s\n", RedColor, err, ResetColor)
            } else {
                fmt.Printf("%s[%s] Parsed RFC5424 Message:%s\n%s%+v%s\n", GreenColor, timestamp, ResetColor, GreenColor, parsedMsg, ResetColor)
            }
        } else {
            fmt.Printf("%s[%s] Message from %v:%s\n%s%s%s\n", GreenColor, timestamp, conn.RemoteAddr(), ResetColor, GreenColor, message, ResetColor)
        }
    }

4. parseRFC5424Message(msg string)

Decodes syslog messages formatted per RFC5424.

func parseRFC5424Message(msg string) (*rfc5424.SyslogMessage, error) {
    parser := rfc5424.NewParser()
    parsedMsg, err := parser.Parse([]byte(msg))
    if err != nil {
        return nil, fmt.Errorf("error parsing RFC5424 message: %w", err)
    }

    // Type assertion to convert syslog.Message to *rfc5424.SyslogMessage
    rfc5424Msg, ok := parsedMsg.(*rfc5424.SyslogMessage)
    if !ok {
        return nil, fmt.Errorf("parsed message is not of type *rfc5424.SyslogMessage")
    }

    return rfc5424Msg, nil

Tying it all together

Here’s the full main.go file:

package main

import (
    "fmt"
    "log"
    "net"
    "os"
    "strings"
    "time"

    "github.com/influxdata/go-syslog/v3/rfc5424"
)

const (
    SYSLOG_PORT = ":6601"
    BUFFER_SIZE = 8192
    BANNER      = `
 __          __  _   _                _       
 \ \        / / | | | |              | |      
  \ \  /\  / /__| |_| |__   ___ _ __| |_   _ 
   \ \/  \/ / _ \ __| '_ \ / _ \ '__| | | | |
    \  /\  /  __/ |_| | | |  __/ |  | | |_| |
     \/  \/ \___|\__|_| |_|\___|_|  |_|\__, |
                                        __/ |
                                       |___/ 
    Syslog Server v1.0.0
    ==========================================
`
    CyanColor   = "\033[1;36m"
    GreenColor  = "\033[1;32m"
    RedColor    = "\033[1;31m"
    YellowColor = "\033[1;33m"
    ResetColor  = "\033[0m"
)

type RFC5424Message struct {
    Priority       int
    Version        string
    Timestamp      time.Time
    Hostname       string
    AppName        string
    ProcID         string
    MsgID          string
    StructuredData string // New field for structured data
    Message        string
}

func parseRFC5424Message(msg string) (*rfc5424.SyslogMessage, error) {
    parser := rfc5424.NewParser()
    parsedMsg, err := parser.Parse([]byte(msg))
    if err != nil {
        return nil, fmt.Errorf("error parsing RFC5424 message: %w", err)
    }

    // Type assertion to convert syslog.Message to *rfc5424.SyslogMessage
    rfc5424Msg, ok := parsedMsg.(*rfc5424.SyslogMessage)
    if !ok {
        return nil, fmt.Errorf("parsed message is not of type *rfc5424.SyslogMessage")
    }

    return rfc5424Msg, nil
}

func printStartupInfo() {
    fmt.Print(CyanColor)
    fmt.Print(BANNER)
    fmt.Print(ResetColor)

    hostname, err := os.Hostname()
    if err != nil {
        hostname = "unknown"
    }

    fmt.Print(GreenColor)
    fmt.Printf("🚀 Starting Wetherly Syslog Server...\n")
    fmt.Printf("📅 Time: %s\n", time.Now().Format(time.RFC1123))
    fmt.Printf("💻 Hostname: %s\n", hostname)
    fmt.Printf("🔌 Protocol: TCP\n")
    fmt.Printf("🎯 Port: 6601\n")
    fmt.Printf("📦 Buffer Size: %d bytes\n", BUFFER_SIZE)
    fmt.Print(ResetColor)
    fmt.Println("==========================================")
}

func main() {
    printStartupInfo()

    listener, err := net.Listen("tcp", SYSLOG_PORT)
    if err != nil {
        log.Fatalf("%sError creating TCP listener: %v%s", RedColor, err, ResetColor)
    }
    defer listener.Close()

    fmt.Printf("%s✅ Server is ready to accept connections%s\n\n", GreenColor, ResetColor)

    for {
        conn, err := listener.Accept()
        if err != nil {
            log.Printf("%sError accepting connection: %v%s", RedColor, err, ResetColor)
            continue
        }

        fmt.Printf("%s📥 New connection from %s%s\n", GreenColor, conn.RemoteAddr(), ResetColor)
        go handleConnection(conn)
    }
}

func handleConnection(conn net.Conn) {
    defer conn.Close()

    buffer := make([]byte, BUFFER_SIZE)
    for {
        n, err := conn.Read(buffer)
        if err != nil {
            if err.Error() != "EOF" {
                log.Printf("%sError reading from connection: %v%s", RedColor, err, ResetColor)
            }
            fmt.Printf("%s📤 Connection closed from %s%s\n", YellowColor, conn.RemoteAddr(), ResetColor)
            return
        }

        message := string(buffer[:n])
        timestamp := time.Now().Format("2006-01-02 15:04:05")

        if strings.HasPrefix(message, "<") {
            parsedMsg, err := parseRFC5424Message(message)
            if err != nil {
                fmt.Printf("%sError parsing RFC5424 message: %v%s\n", RedColor, err, ResetColor)
            } else {
                fmt.Printf("%s[%s] Parsed RFC5424 Message:%s\n%s%+v%s\n", GreenColor, timestamp, ResetColor, GreenColor, parsedMsg, ResetColor)
            }
        } else {
            fmt.Printf("%s[%s] Message from %v:%s\n%s%s%s\n", GreenColor, timestamp, conn.RemoteAddr(), ResetColor, GreenColor, message, ResetColor)
        }
    }
}

Testing and Deployment

Once our server is built, it's time to test and deploy it. We use Docker to containerize our application, ensuring it runs consistently across different environments. The Dockerfile is straightforward, building our Go application and packaging it into a lightweight Alpine image.

Running the Server

To run the server locally, use the following command:

docker run -p 6601:6601 jleski/wetherly:latest

This will start the server, ready to accept syslog messages on port 6601.

Sending Test Messages

We can send test messages using Netcat or the task command. For example, to send a simple test message, use:

task test:send

For more complex messages, such as those formatted according to RFC5424, use:

task test:send:rfc5424

Check my GitHub repository for the full code: jleski/wetherly: Syslog receiver

Conclusion

And there you have it—a mostly harmless syslog server, ready to log messages from across the digital cosmos. As you continue to explore and enhance this codebase, remember the words of Douglas Adams: "Don't Panic." With a well-prepared task list and a touch of humor, you're well-equipped to tackle any challenge that comes your way. May your logs be ever verbose, your errors be few, and your adventures be plentiful.

0
Subscribe to my newsletter

Read articles from Jaakko Leskinen directly inside your inbox. Subscribe to the newsletter, and don't miss out.

golangsyslogProgramming BlogsDevopsDockerKubernetessoftware developmentloggingOpen Source

Written by

Jaakko Leskinen
Jaakko Leskinen