Build a URL Shortener with Go Fiber and Redis

Before we dive into the implementation, make sure you have the following prerequisites installed:

Go programming language & Redis server

Step 1: Setting Up the Go Fiber Project & Defineing Routes. Start by creating a new Go Fiber project. Use the following commands in your terminal:

go mod init github.com/suraj/url-shortener
go get -u github.com/gofiber/fiber/v2

You can replace (github.com/suraj/url-shortener) as per your project name.

Firstly create a folder structure for the projects.

//Folder structure
Go-projects/
|-- api/
|   |-- helpers/
|   |   |-- helpers.go
|   |
|   |-- routes/
|       |-- resolve.go
|       |-- shortren.go
|
|-- internal/
|   |-- database/
|       |-- database.go
|
|-- .env
|-- main.go
|-- Makefile

Create a .env file in the root folder, for secret credentials

//.env
DB_HOST="example"
DB_PORT="example"
DB_PASS="example"

Create a new folder named 'internal'. Inside 'internal', create a folder named 'database'. Inside 'database', create a file named 'database.go' and import the necessary packages.

//database.go
package database

import (
    "context"
    "fmt"
    "os"
    "sync"

    "github.com/go-redis/redis"
)

var (
    ctx          = context.TODO()
    redisConnMap = make(map[int]*redis.Client)
    redisConnMu  sync.Mutex
)

func CreateRedisClient(dbNo int) *redis.Client {
    redisConnMu.Lock()
    defer redisConnMu.Unlock()

    if client, ok := redisConnMap[dbNo]; ok {
        return client
    }

    redisConn := redis.NewClient(&redis.Options{
        Addr:     fmt.Sprintf("%s:%s", os.Getenv("DB_HOST"), os.Getenv("DB_PORT")),
        Password: os.Getenv("DB_PASS"),
        DB:       dbNo,
    })

    if _, err := redisConn.Ping().Result(); err != nil {
        return nil
    }

    redisConnMap[dbNo] = redisConn
    return redisConn
}

Create a new file named main.go and import the necessary packages:

//main.go
package main

import (
    "log"
    "os"

    "github.com/joho/godotenv"

    "github.com/gofiber/fiber/v2"
    "github.com/gofiber/fiber/v2/middleware/logger"
    "github.com/suraj/url-shortener/api/routes"
)

func setupRoutes(app *fiber.App) {
    app.Get("/:url", routes.ResolveURL)
    app.Post("/api/v1", routes.ShortenURL)
}

func main() {

    if err := godotenv.Load(); err != nil {
        log.Fatal("Error loading .env file")
    }

    app := fiber.New(fiber.Config{})

    app.Use(logger.New())

    setupRoutes(app)

    address := ":" + os.Getenv("APP_PORT")

    log.Fatal(app.Listen(address))
}

Step 2: Create URL Shortener Logic:

To define the routes for your URL shortener, you should create a file. For this example, we'll be creating two endpoints: one for shortening URLs and another for resolving short URLs to their original counterparts. To begin, create a folder named 'API'. Within this folder, create two additional sub-folders named 'helpers' and 'routes'.

In the helpers folder create file helpers.go, it is used for command functions that can used repeatedly.

//helpers.go
package helpers

import (
    "os"
    "strings"
)

func EnforceHTTP(url string) string {
    if !strings.HasPrefix(url, "http") {
        return "http://" + url
    }
    return url
}

func RemoveDomainError(url string) bool {
    url = strings.TrimPrefix(url, "http://")
    url = strings.TrimPrefix(url, "https://")
    url = strings.TrimPrefix(url, "www.")
    return strings.SplitN(url, "/", 1)[0] != os.Getenv("DOMAIN")
}

In the routes folder create two files resolver.go and shortener.go in this file we will write logic for our URL shortener

//resolver.go 
package routes

import (
    "github.com/go-redis/redis"
    "github.com/gofiber/fiber/v2"
    "github.com/suraj/url-shortener/internal/database"
)

const (
    notFoundStatus      = fiber.StatusNotFound
    internalErrorStatus = fiber.StatusInternalServerError
    redirectStatusCode  = 301
    redisDatabaseMain   = 0
    redisDatabaseIncr   = 1
)

func ResolveURL(c *fiber.Ctx) error {
    url := c.Params("url")

    r := database.CreateRedisClient(redisDatabaseMain)
    defer r.Close()

    value, err := r.Get(url).Result()
    switch {
    case err == redis.Nil:
        return c.Status(notFoundStatus).JSON(fiber.Map{
            "error": "Short not found in the database",
        })
    case err != nil:
        return c.Status(internalErrorStatus).JSON(fiber.Map{
            "error": "Cannot connect to the database",
        })
    }

    _ = r.Incr("counter")

    return c.Redirect(value, redirectStatusCode)
}

---

//shortener.go
package routes

import (
    "os"
    "strconv"
    "time"

    "github.com/asaskevich/govalidator"
    "github.com/go-redis/redis"
    "github.com/gofiber/fiber/v2"
    "github.com/google/uuid"
    "github.com/suraj/url-shortener/api/helpers"
    "github.com/suraj/url-shortener/internal/database"
)

type request struct {
    URl         string        `json:"url"`
    CustomShort string        `json:"short"`
    Expiry      time.Duration `json:"expiry"`
}

type response struct {
    URL            string        `json:"url"`
    CustomShort    string        `json:"short"`
    Expiry         string        `json:"expiry"`
    XRateRemaining int           `json:"rate_limit"`
    XRateLimitRest time.Duration `json:"rate_limit_reached"`
}

const (
    defaultExpiry      = 24 * time.Hour
    apiQuotaTTL        = 30 * time.Minute
    defaultRateLimit   = 10
    rateLimitDecrement = 1
)

func ShortenURL(c *fiber.Ctx) error {
    body := new(request)

    if err := c.BodyParser(&body); err != nil {
        return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{"error": "Invalid request body"})
    }

    r := database.CreateRedisClient(redisDatabaseMain)
    defer r.Close()

    r2 := database.CreateRedisClient(redisDatabaseIncr)
    defer r2.Close()

    // Rate Limiting
    val, err := r2.Get(c.IP()).Result()
    if err == redis.Nil {
        _ = r2.Set(c.IP(), os.Getenv("API_QUOTA"), apiQuotaTTL).Err()
    } else {
        valInt, _ := strconv.Atoi(val)
        if valInt <= 0 {
            limit, _ := r2.TTL(c.IP()).Result()
            return c.Status(fiber.StatusServiceUnavailable).JSON(fiber.Map{
                "error":           "Rate limit exceeded",
                "rate_limit_rest": limit / time.Minute,
            })
        }
    }

    // Validate URL
    if !govalidator.IsURL(body.URl) || !helpers.RemoveDomainError(body.URl) {
        return c.Status(fiber.StatusBadRequest).JSON(fiber.Map{"error": "Invalid URL"})
    }

    body.URl = helpers.EnforceHTTP(body.URl)

    // Generate Short URL ID
    var id string
    if body.CustomShort == "" {
        id = uuid.New().String()[:6]
    } else {
        id = body.CustomShort
    }

    // Check if Short URL ID already exists
    val, _ = r.Get(id).Result()
    if val != "" {
        return c.Status(fiber.StatusForbidden).JSON(fiber.Map{"error": "Short URL already exists"})
    }

    // Set Short URL in Redis
    if body.Expiry == 0 {
        body.Expiry = time.Duration(defaultExpiry.Hours()) * time.Hour
    }

    err = r.Set(id, body.URl, time.Duration(body.Expiry)*time.Hour).Err()
    if err != nil {
        return c.Status(fiber.StatusInternalServerError).JSON(fiber.Map{"error": "Unable to connect to server"})
    }

    // Response
    resp := response{
        URL:            body.URl,
        CustomShort:    "",
        Expiry:         time.Duration(body.Expiry).String(),
        XRateRemaining: defaultRateLimit,
        XRateLimitRest: apiQuotaTTL / time.Minute,
    }

    // Update Rate Limiting
    r2.DecrBy(c.IP(), rateLimitDecrement)

    val, _ = r2.Get(c.IP()).Result()
    resp.XRateRemaining, _ = strconv.Atoi(val)

    ttl, _ := r2.TTL(c.IP()).Result()
    resp.XRateLimitRest = ttl / time.Minute

    resp.CustomShort = os.Getenv("DOMAIN") + "/" + id

    return c.Status(fiber.StatusOK).JSON(resp)
}

Optional - Makefile, If you want to use it.

//Makefile
# Go parameters
GOCMD=go
GOBUILD=$(GOCMD) build
GOCLEAN=$(GOCMD) clean
GOTEST=$(GOCMD) test
GOGET=$(GOCMD) get

# Binary name
BINARY_NAME=url-shortener

# Main build target
all: test build

# Build the binary
build:
    $(GOBUILD) -o $(BINARY_NAME) -v

# Run tests
test:
    $(GOTEST) -v ./...

# Clean up
clean:
    $(GOCLEAN)
    rm -f $(BINARY_NAME)

# Install dependencies
deps:
    $(GOGET) -u github.com/gofiber/fiber/v2
    $(GOGET) -u gorm.io/gorm/logger
    $(GOGET) -u github.com/joho/godotenv
    $(GOGET) -u github.com/go-redis/redis/v8
    $(GOGET) -u github.com/asaskevich/govalidator
    $(GOGET) -u github.com/go-delve/delve/cmd/dlv

# Run the application
run:
    $(GOBUILD) -o $(BINARY_NAME) -v ./...
    ./$(BINARY_NAME)

#background dependencies
tidy:
    $(GOCMD) mod tidy

# Shortcuts
.PHONY: all build test clean deps run tidy

Now the Coding part for the URL Shortener has been completed, you can check by using Postman that it working properly like the given image below

Conclusion:

You have scratched the basics of Golang with this project, you need to explore more topics like Concurrency Patterns, Generics, Reflection and many more.

Thanks For Reading This Blog.