Packaging Clojure projects into jars and uberjars with tools.build

Article describes how to create a jar or uberjar using tools.build library. How to create build.clj file with build functions and create alias in deps.edn to invoke build functions via Clojure CLI

Video version of this article

Jars and uberjars

The most common way to prepare your Clojure project for distribution is to pack it into a *.jar file.

JAR format came from the Java world and it stands for "Java ARchive". It is a zip archive with a *.jar extension that contains Java class files, resources, and metadata.

To distribute Clojure library you can create a jar with library's source files or with compiled code (Java *.class files with bytecode). Optionally you can put both - source files and compiled files. Compiled files will be preferred over a source files unless source files are newer.

To distribute a Clojure application you have to create a jar that contains compiled Clojure code of the app along with all its dependencies. Such self-contained archive called uberjar.

jar/uberjar content

In addition to Clojure sources and Java bytecode files, jars can contain resources and metadata files.

Resouces

When preparing the jar you can put there any resource files that your library or app needs (images, text, etc). From the Clojure code, those resources can be accessed via (io/resource) function.

MANIFEST.mf

Manifest is a special file that contains information about jar content. Here can be specified the entry point of an application but you won't need to do that manually. tools.build will generate a default manifest for every jar automatically. If you will need to extend manifest with custom fields, you can use :manifest option in (jar) and (uber) functions of tools.build.

More info about manifest can be found in Java documentation.

pom.xml

In the Java world, there is a super-popular buiid framework called Maven. it uses pom.xml files to store project information and configuration. This framework also has its own type of repository for built projects called "maven repository".

In Clojure, we don't have to use Maven to build projects but we extensively use maven repositories to store build artifacts (jars). Dependencies of our project could be downloaded from maven repos and the resulting artifact of our project (its jar) can be uploaded to maven repo to be accessible by others.

The most popular maven repo for Clojure artifacts is Clojars.

If you are planning to upload your project's jar to some maven repository, that jar should contain a pom.xml with artifact info.

More info about POM can be found in maven documentation.

tools.build library

Not so long ago you had to use 3rd-party tools to create jars for your own Clojure projects. But now the official tools.build library can be used.

The main idea behind tools.build is that project's build is also a program and it can be written in Clojure code. And tools.build is a library that provides functions commonly needed for builds.

adding tools.build to a project

Let's imagine that we have a simple project structured like this:

project
├── deps.edn
└── src
    └── ...

To start using tools.build we need to:

• create a Clojure namespace to keep build functions
• add a new alias to deps.edn to call build functions with its help
• invoke build function via Clojure CLI

Let's add a build.clj to the root of the project. This is where we are going to keep all the build tasks. For now, there will be only one function - (clean) . It removes target directory using (b/delete) from tools.build.api. Inside target, we will keep all our build artifacts, so it should be cleaned between builds.

We don't use any arguments inside (clean) but it should be defined as a function with one argument because, when invoked from Clojure CLI, it will be called with one - map of the arguments passed via the command line (nil if you didn't pass any).

build.clj:

(ns build
  (:require [clojure.tools.build.api :as b]))                     ; requiring tools.build

(def build-folder "target")

(defn clean [_]
  (b/delete {:path build-folder})                                 ; removing artifacts folder with (b/delete)
  (println (format "Build folder \"%s\" removed" build-folder)))

To make build functions accessible via Clojure CLI, let's create a new alias in deps.edn. In that alias there should be :deps key with tools.build dependency and :ns-default key which tells CLI in what namespace it should look for the function mentioned in the command line.

deps.edn:

{
; ... other deps.edn content ...
:aliases
  {
    ; ... other aliases ...

    ; build alias:
    :build {:deps {io.github.clojure/tools.build {:git/tag "v0.8.1" :git/sha "7d40500"}}
            :ns-default build}}  ; <-- set build namespace as default
}

And finally, to run (clean) from the command line using the new alias, we need to call Clojure CLI with -T option. That option ignores the rest of deps.edn content and uses the dependencies only from the current alias. In this way, we do not interfere with the other project dependencies and source files.

$ clj -T:build clean
Build folder "target" removed

creating a jar for the library

Now let's review an example of creating a jar for the library with the following folder structure

mymathlibary
├── build.clj
├── deps.edn
├── resources
│   └── lib_resource.txt
└── src
    └── mymath
        └── sum.clj

Inside sum.clj there is a sum function that adds two values and prints text from the resource file (please promise me to not do like this in real-world libs).

sum.clj:

(ns mymath.sum
  (:require [clojure.java.io :as io]))

(defn sum [a b]
  (println "Lib resource content:" (slurp (io/resource "mymath_resource.txt")))
  (+ a b))

Text inside lib_resource.txt:

hello from mymath lib

Let's say we want our jar to contain only source code, without compiled bytecode. To achieve this, in build.clj we need to add a new function named (jar) that does the following:

• cleans target directory from leftovers
• copies sources and resources to target. They should go into the result jar.
• creates pom.xml file. You will need it if you are going to put the library into maven repositories.
• creates the jar file

Here is the code of our new build.clj:

(ns build
  (:require [clojure.tools.build.api :as b]))

(def build-folder "target")
(def jar-content (str build-folder "/classes"))     ; folder where we collect files to pack in a jar

(def lib-name 'com.github.YOURNAME/mymath-lib)      ; library name
(def version "0.0.1")                               ; library version
(def basis (b/create-basis {:project "deps.edn"}))  ; basis structure (read details in the article)
(def jar-file-name (format "%s/%s-%s.jar" build-folder (name lib-name) version))  ; path for result jar file

(defn clean [_]
  (b/delete {:path build-folder})
  (println (format "Build folder \"%s\" removed" build-folder)))

(defn jar [_]
  (clean nil)                                     ; clean leftovers

  (b/copy-dir {:src-dirs   ["src" "resources"]    ; prepare jar content
               :target-dir jar-content})

  (b/write-pom {:class-dir jar-content            ; create pom.xml
                :lib       lib-name
                :version   version
                :basis     basis
                :src-dirs  ["src"]})

  (b/jar {:class-dir jar-content                  ; create jar
          :jar-file  jar-file-name})
  (println (format "Jar file created: \"%s\"" jar-file-name)))

In the code of (jar) we use three functions from tools.build to achieve steps mentioned before: (b/copy-dir), (b/write-pom) and (b/jar). Their names and arguments are pretty self-explanatory except one - :basis.

The basis is a big structure that contains a superset of all deps.edn files, project classpath, and description of all dependencies. In the official Clojure documentation, it is mentioned here.

tools.build uses basis in a few functions and gives a function to create it - (b/create-basis). The (b/write-pom) function uses basis to correctly reflect the library dependencies in pom.xml

Now to create the jar we can run (jar) function using Clojure CLI (assuming your deps.edn already contains :build alias):

$ clj -T:build jar
Build folder "target" removed
Jar file created: "target/mymath-lib-0.0.1.jar"

Now it can be uploaded to a maven repo and used as a dependency in other projects.

You can examine jar content using jar -tf command:

$ jar -tf target/mymath-lib-0.0.1.jar 
META-INF/MANIFEST.MF
lib_resource.txt
META-INF/
mymath/
META-INF/maven/
mymath/sum.clj
META-INF/maven/com.github.YOURNAME/
META-INF/maven/com.github.YOURNAME/mymath-lib/
META-INF/maven/com.github.YOURNAME/mymath-lib/pom.xml
META-INF/maven/com.github.YOURNAME/mymath-lib/pom.properties

creating a runnable ubjerjar

Now we can take a look at how to create a runnable uberjar for a Clojure application.

For example, we have a project with the following structure:

simpleapp
├── build.clj
├── deps.edn
├── resources
│   └── app_resource.txt
└── src
    └── dev
        └── core.clj

core.clj contains (-main) function that will work as an entry point when the application invoked. The (-main) calls (sum) function from the library created in a previous section and prints text from the local resource file.

core.clj:

(ns dev.core
  (:require [mymath.sum :as s]
            [clojure.java.io :as io])
  (:gen-class))     ; !!! instruction to generate bytecode for java class from this namespace

(defn -main []
  (println "Sum is:" (s/sum 1 2))
  (println "App resource content:" (slurp (io/resource "app_resource.txt"))))

Most important thing here is a :gen-class directive to (ns) function. It tells Clojure compiler to generate an additional bytecode file with java class corresponding to this namespace. That class will serve an entry point for our uberjar.

Now in build.clj we can create an (uber) function that builds uberjar. Here is what it should do:

• clean target directory from leftovers
• copy resources to target. They should go into result uber.
• compile Clojure code
• create the uberjar with (-main) entrypoint

Full code of build.clj:

(ns build
  (:require [clojure.tools.build.api :as b]))

(def build-folder "target")
(def jar-content (str build-folder "/classes"))

(def basis (b/create-basis {:project "deps.edn"}))
(def version "0.0.1")
(def app-name "myapp")
(def uber-file-name (format "%s/%s-%s-standalone.jar" build-folder app-name version)) ; path for result uber file

(defn clean [_]
  (b/delete {:path "target"})
  (println (format "Build folder \"%s\" removed" build-folder)))

(defn uber [_]
  (clean nil)

  (b/copy-dir {:src-dirs   ["resources"]         ; copy resources
               :target-dir jar-content})

  (b/compile-clj {:basis     basis               ; compile clojure code
                  :src-dirs  ["src"]
                  :class-dir jar-content})

  (b/uber {:class-dir jar-content                ; create uber file
           :uber-file uber-file-name
           :basis     basis
           :main      'dev.core})                ; here we specify the entry point for uberjar

  (println (format "Uber file created: \"%s\"" uber-file-name)))

The code is quite similar to the one where we were creating jar and probably doesn't require many explanations. New functions here are (b/compile-clj) and (b/uber).

By the way you could have used (b/compile-clj) in library example to create a library jar with compiled code.

Now let's bulid our uberjar:

$ clj -T:build uber
Build folder "target" removed
Uber file created: "target/myapp-0.0.1-standalone.jar"

And run it using java -jar command:

$ java -jar target/myapp-0.0.1-standalone.jar
Lib resource content: hello from mymath lib
Sum is: 3
App resource content: hello from app

As we can see, resource file content was printed from the app and the lib. That means uberjar was created successfully and contains all the necessary files.

The output of jar -tf target/myapp-0.0.1-standalone.jar will be too long to post it here but I encourage you to investigate the content of created uberjar to see how dependencies are included there.

more tools.build functions

Your build.clj is not restricted to creating jars and uberjars. There are more functions in tools.build that give you freedom to build scripts of an arbitrary complexity:

(process) - runs an arbitrary command with arguments

(git-process) - runs git command

(zip) and (unzip) - work with archives

(install) - installs jar to a local maven repository

More details can be found in the tools.build official api.