go · golang · working directory · relative

External Assets, Working Directories, and Go

I think about working directories and file paths almost every time I work on a Go project.

Accessing external files from Go code seems like a straightforward task at first glance. However, once you are distributing your program, you can’t be sure where your users are running it from or where they are storing it on their file system. Keeping track of your external assets can become a problem at that point.

Using External Assets with Go

Consider the following main.go that starts a simple HTTPS server:

package main

import (
    "fmt"
    "log"
    "net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hi there!")
}

func main() {
    http.HandleFunc("/", handler)
    err := http.ListenAndServeTLS(":8081", "cert.pem", "key.pem", nil)
    log.Println(err)
}

Simple, right? We need a server certificate and a private key to start the HTTPS server. That data is loaded from the provided “cert.pem” and “key.pem” files.

Now think about where those files need to be located so that, once we compile the above program, the resulting Go binary can find them. We provided two file paths: “cert.pem” and “key.pem”.

Our Go binary will look for them in the working directory. Here’s an example:

Our compiled HTTPS server binary is saved as /home/myuser/https-test/server, the server certificate and private key files are saved as /home/myuser/https-test/cert.pem and /home/myuser/https-test/key.pem.

Now, using the Terminal, we change into that directory with

cd /home/myuser/https-test

and execute the server binary:

./server

Great! Our program is working as expected. It reads the certificate and key files and starts a HTTPS server on port 8081.

Let’s try that again, but this time we’ll start the binary from another directory:

cd /home/myuser

After that we’ll start the server with:

https-test/server

But this time we are presented with an error:

open cert.pem: no such file or directory

Why is that? Well, our server binary is looking for “cert.pem” (and “key.pem”) in its working directory. Since we called the binary from /home/myuser, that directory is now the working directory. And because there is no /home/myuser/cert.pem, our program throws an error.

How do we fix that?

Some Go programmers argue that including all assets into the binary is the way to go. That way distributing your program is as easy as copying the binary. Additionally, a simple go build will place a runnable program into your $GOPATH/bin directory.

That’s fine as long as those assets should not be provided by the user or you want to keep your binary as lean as possible.

I usually want my asset files to be external and located in a directory relative to the binary.

Setting the Working Directory at Runtime

To achieve that, we need the Go binary to be aware of the path it is in. Luckily, Daniel Theophanes created a library that helps us with that. It’s called osext.

Let’s create a new Go source file next to the “main.go”. We’ll call it “filenames.go” and it will hold the following code:

package main

import (
    "github.com/kardianos/osext"
    "log"
    "os"
)

var (
    // Initialization of the working directory. Needed to load asset files.
    _ = determineWorkingDirectory()
    // File names for the HTTPS certificate
    certFilename = "cert.pem"
    keyFilename  = "key.pem"
)

func determineWorkingDirectory() string {
    // Get the absolute path this executable is located in.
    executablePath, err := osext.ExecutableFolder()
    if err != nil {
        log.Fatal("Error: Couldn't determine working directory: " + err.Error())
    }
    // Set the working directory to the path the executable is located in.
    os.Chdir(executablePath)
    return ""
}

After that, we just have to change err := http.ListenAndServeTLS(":8081", "cert.pem", "key.pem", nil) in the “main.go” file to

err := http.ListenAndServeTLS(":8081", certFilename, keyFilename, nil)

And that’s it. If we now run the compiled binary from a directory other than its own, it’ll work just fine.

How did we do that? Let’s look at the source code in “filenames.go”.

The way Go works, variables are always determined right after importing the packages. After that, the init function is called and only after the init function terminates does the main function execute. In chronological order:

import -> var -> init() -> main()

That is why we are able to determine the directory our executable is in by using the imported osext package. We then set the working directory to that path by calling os.Chdir(executablePath).

After that, our HTTPS server can safely use certFilename and keyFilename to load the certificate data.

Sometimes though, you want to give your users the option to provide a custom path to load assets from.

You might have noticed that the determineWorkingDirectory function returns an empty string that is not used anywhere. I did that so we can easily change the function to accommodate a user-specified asset path.

It’s a good idea to do that in your program. When using a Docker container for example you could point the asset path to a persistent directory so that your files survive a restart.

Letting the User Specify a Custom Path

To do that, we need to change the code in “filenames.go” in the following way:

package main

import (
    "flag"
    "github.com/kardianos/osext"
    "log"
    "os"
    "path/filepath"
)

var (
    // Initialization of the working directory. Needed to load asset files.
    filePath = determineWorkingDirectory()
    // File names for the HTTPS certificate
    certFilename = filepath.Join(filePath, "cert.pem")
    keyFilename  = filepath.Join(filePath, "key.pem")
)

func determineWorkingDirectory() string {
    var customPath string
    // Check if a custom path has been provided by the user.
    flag.StringVar(&customPath, "custom-path", "", "Specify a custom path to the asset files. This needs to be an absolute path.")
    flag.Parse()
    // Get the absolute path this executable is located in.
    executablePath, err := osext.ExecutableFolder()
    if err != nil {
        log.Fatal("Error: Couldn't determine working directory: " + err.Error())
    }
    // Set the working directory to the path the executable is located in.
    os.Chdir(executablePath)
    // Return the user-specified path. Empty string if no path was provided.
    return customPath
}

As you can see, we extended the determineWorkingDirectory function to parse the command line flag “custom-path” that the user can use to provide a custom path to the asset files.

After evaluating the filePath variable, we prepend that path to the “cert.pem” and “key.pem” file name variables. Here we are using the filepath.Join method to create a path string according to the current operating system (e. g. using forward slashes for Unix, backslashes for Windows).

If the user didn’t provide a custom path, the filePath variable will be an empty string. That way, our program behaves just as it did before: It will look for the”cert.pem” and “key.pem” files in the working directory.

Since variables are always evaluated before the main function, we can then safely use certFilenameand keyFilename to start the HTTPS server in “main.go”.

Conclusion

Thinking about file locations is essential when working with external assets. Luckily, when using Go, we have enough tools in our hands to keep track of assets regardless of where the binary is located and which operating system it is running on.

However, be aware that by using the above method you are essentially overwriting the default behavior of the OS. An educated user might count on the fact that the directory he is calling the binary from will be its working directory.

At the end of the day, there is no right way to hande assets that covers all bases. I would argue that keeping assets in a path relative to your binary makes the distribution of your program easy. Additionally, giving your users the option to provide a custom path to the asset files is considerate and easily done with Go.

Published:
comments powered by Disqus