How not to reveal secrets in logs

In Go, most of the devs use strings to store the secrets which later on can be seen in the log files or can be sent to some external logging tools. It just takes fmt or zap to see the secrets in log. Many incidents have already occurred saying user details were leaked due to the storage of secrets in plain text. Few of the examples were registered in GitHub, Twitter, Justalk and many more.

There always exist a scope that someday someone would store the secrets in plain text, maybe a new joiner or someone working near the deadline may use it to ease his job(you never know). So there should be some mechanism that will be a standard way to store secrets and won't reveal the secrets so easily.

Here is a code that stores secrets in string

package main

import (
    "encoding/json"
    "fmt"

    "go.uber.org/zap"
)

type TokenerStringType struct {
    mainURL  string
    APItoken string
}

func main() {
    logger, _ := zap.NewProduction()
    defer logger.Sync() // flushes buffer, if any
    sugar := logger.Sugar()

    fmt.Println(`Using string to store secrets: TokenerStringType{mainURL: "measutosh.hashnode.dev", APItoken: "mytoken"}`)
    resultString := TokenerStringType{mainURL: "measutosh.hashnode.dev", APItoken: "mytoken"}
    fmt.Printf("fmt.Println result includes the secret: %s\n", resultString)
    b, _ := json.Marshal(resultString)
    fmt.Printf("json.Marshal reveals the secret: %s\n", string(b))
    sugar.Info("zap reveals the secrets", resultString)
}

And result as follows, the token is completely visible in the terminal.

Using string to store secrets: TokenerStringType{mainURL: "measutosh.hashnode.dev", APItoken: "mytoken"}
fmt.Println result includes the secret: {measutosh.hashnode.dev mytoken}
json.Marshal reveals the secret: {"APItoken":"mytoken"}
{"level":"info","ts":1257894000,"caller":"sandbox1313230561/prog.go:25","msg":"zap logging reveals the secrets{measutosh.hashnode.dev mytoken}"}

Alternative to the current method

Instead of using string type we can use custom made types having properties like string. Making it simple by showing an example below.

type Tokener struct {    
    mainURL string    
    APItoken string
}

The above code was the way we store/share secrets. To replace the string type we will be using our own datatypes.

type Tokener struct {    
    mainURL urlString    
    APItoken tokenString
}

Following this we can declare the custom datatypes as per our choice but mostly relative to string type only, kind of an abstraction.

type Tokener struct {    
    mainURL urlString    
    APItoken tokenString
}
type urlString struct {    
    Value string    
}
type tokenString struct {    
    Value string    
}

Here comes the interesting part, how these 2 structs will utilize Stringer and MarshalJSON interfaces.

type Tokener struct {    
    mainURL urlString    
    APItoken tokenString
}

type urlString struct {    
    Value string    
}

type tokenString struct {    
    Value string    
}

func (s urlString) String() string {
  return s.Value
}

func (s urlString) MarshalJSON() ([]byte, error){
  return json.Marshal(s.String())
}

func (s tokenString) String() string {
  return "*****"
}

func (s tokenString) MarshalJSON() ([]byte, error) {    
    return json.Marshal(s.String())
}

Now, the redacted string "****" or JSON both will make the job difficult to understand even if the secrets get leaked in the log files. Here is the full code

package main

import (
    "encoding/json"
    "fmt"

    "go.uber.org/zap"
)

type urlString struct {
    Value string
}

type tokenString struct {
    Value string
}

func (s urlString) String() string {
    return s.Value
}
func (s urlString) MarshalJSON() ([]byte, error) {
    return json.Marshal(s.String())
}

func (s tokenString) String() string {
    return "*****"
}
func (s tokenString) MarshalJSON() ([]byte, error) {
    return json.Marshal(s.String())
}

type TokenerCustomType struct {
    mainURL  urlString
    APItoken tokenString
}

type TokenerStringType struct {
    mainURL  string
    APItoken string
}

func main() {
    logger, _ := zap.NewProduction()
    defer logger.Sync() // flushes buffer, if any
    sugar := logger.Sugar()

    fmt.Println(`Using String type: TokenerStringType{mainURL: "measutosh.hashnode.dev", APItoken: "mytoken"}`)
    resultString := TokenerStringType{mainURL: "measutosh.hashnode.dev", APItoken: "mytoken"}
    fmt.Printf("fmt.Println result includes the secret: %s\n", resultString)
    b, _ := json.Marshal(resultString)
    fmt.Printf("json.Marshal reveals the secret: %s\n", string(b))
    sugar.Info("zap reveals the secrets", resultString)
    fmt.Printf("\n\n")

    fmt.Println(`Using Custom type: TokenerCustomType{mainURL: urlString{"measutosh.hashnode.dev"}, APItoken: tokenString{"mytoken"}}`)
    resultStringType := TokenerCustomType{mainURL: urlString{"measutosh.hashnode.dev"}, APItoken: tokenString{"secret"}}
    fmt.Printf("fmt.Println result includes the redacted secrets: %s\n", resultStringType)
    b, _ = json.Marshal(resultStringType)
    fmt.Printf("json.Marshal couldn't reveal secrets: %s\n", string(b))
    sugar.Info("zap logging couldn't reveal secrets", resultStringType)
}

And the result goes like this

Using String type: TokenerStringType{mainURL: "measutosh.hashnode.dev", APItoken: "mytoken"}
fmt.Println result includes the secret: {measutosh.hashnode.dev mytoken}
json.Marshal reveals the secret: {"APItoken":"mytoken"}
{"level":"info","ts":1257894000,"caller":"sandbox2607379396/prog.go:52","msg":"zap reveals the secrets{measutosh.hashnode.dev mytoken}"}


Using Custom type: TokenerCustomType{mainURL: urlString{"measutosh.hashnode.dev"}, APItoken: tokenString{"mytoken"}}
fmt.Println result includes the redacted secrets: {{measutosh.hashnode.dev} *****}
json.Marshal couldn't reveal secrets: {"APItoken":"*****"}
{"level":"info","ts":1257894000,"caller":"sandbox2607379396/prog.go:60","msg":"zap logging couldn't reveal secrets{{measutosh.hashnode.dev} *****}"}

Now, you can see that the token is only visible in redacted format. Does it end here!The secrets will never be visible in terminal anymore?Yes, it can be but from this method it's crystal clear that what's ans noraml type and what's a secret or custom type that needs extra care. If you want then run the code yourself here.

That's it for now. Thanks😊