Mission Impossible Code - Compact, Idempotent, DevOps Oriented, Multi-Distro Package Installer Script for Linux and Mac

Everyone loves Linux for its ability to stick to fundamentals and common platform expectations. Except those of us who do a lot of deployment automation.

Why? Two main reasons: In a world of stripped back distros (think containers), even fundamental Gnu coreutils and other basics can be missing. Which leads to the second frustration - for some reason distro families thought it would be great to not only innovate package management technology but also change up the command set and also not provide a universal command set (like an api) to hide the differences.

So after bashing my head on this a million times, a bit of bash code eventually emerged (yeah from my head). And as you’ll read, it was not a process of random chance and natural selection - but rather it’s opposite - hyper-engineering.

As per Mission Impossible Code Principles I’ve tried to make it “as simple as possible, but still have a very broad scope of reuse”

FYI - Windows isn’t any better - while Windows PackageManagement (aka OneGet) and Chocolatey both tried to consolidate down to one command set for all package types for the platform, we now have a third one.

TL;DR

I used to take time to expand the section “Architecture Heuristics” and discuss the enablement created by each of the bullets in the section.

I won’t be doing that any longer for these reasons:

1. I have added “Benefits” and “Coding Decisions” to each of the below to provide a synopsis of how the item benefits the solution and what coding choices it affected - which provides a better information map for faster learning.
2. By providing this learning section as mapped bullets - you can incrementally learn a few points by scanning and deep diving what interests you without feeling obligated to plow through paragraphs of prose.
3. It’s quicker and easier for me to document without having to generate formal copy - I’ll be more tempted to share when the authoring process is lighter.

Architecture Heuristics: Requirements, Constraints, Desirements, Serendipities, Applicability, Limitations and Alternatives

This section appears in many Mission Impossible Code samples because it: 1) Demonstrates there is an actual pattern to the thinking underlying this approach, 2) It allows you to learn the patterns but making the thinking plain, 3) It shows the complex engineering of creating simplicity.

The following list demonstrates the Architectural thrust of the solution. This approach is intended to be pure to simplicity of operation and maintenance, rather than purity of a language or framework or development methodology. It is also intended to have the least possible dependencies. It’s an approach I call “Mission Impossible Coding” because it enables the code to get it’s job done no matter what.

• Requirement: (Satisfied) Adhere to “Adoption Driven Development for Tooling”
  • Benefits: everyone will use more of your stuff, more people will truly ’love’ your stuff, driving implementation decision toward “human cognitive” priorities over machine considerations - because we all have much less time than computing resources
  • Coding Decisions: Adhere to Mission Impossible Techniques (aka the entire Architecture Heuristics section)
• Requirement: (Satisfied) Don’t rely on errors to do detections (passive detection)
  • Benefits: creating automation errors when formal exception checking needs to be supported
  • Coding Decisions: picking “command -v” to detect existence of a command
• Requirement: (Satisfied) Don’t require prerequisites to work
  • Benefits: avoid chicken and the egg problem, works on minimalized containers images
  • Coding Decisions: picking “command -v” to detect existence of a command
• Requirement: (Satisfied) Be compact.
  • Benefits: Enables “single script” implementations as the code is not overly obnoxious at the top of a script. Many orchestration systems can send a single script to an endpoint, but sending files makes their scenario much more complex - compact code can travel inside the single script.
  • Coding Decisions: IFS + read method of parsing key value pairs, some single line if statements
• Requirement: (Satisfied) Ensure parameters can be passed in from an unknown number of parent levels and unknown technologies or computing languages
  • Benefits: very broad reuse with no adaptations for many levels of enclosing orchestration.
  • Coding Decisions: Work with a 1) list that is 2) constructed as a string (so that it can be passed between all types of automation technology).
• Requirement: (Satisfied) Handle at least yum and apt-get.
• Requirement: (Satisfied) Be idempotent by checking for the existence of a command before attempting installation operations.
  • Benefits: all the benefits of idempotency which are too many to list here.
• Requirement: (Satisfied) Work for scenarios where the name of the package is different than the name of the command that is needed.
  • Benefits: broader reuse due to handling this frequent use case
  • Coding Decisions: use key value pairs for installing packages to get specific commands
• Requirement: (Satisfied) Auto-detect the package manager.
  • Benefits: broader reuse
• Requirement: (Satisfied) Handle the AWFUL problem of brew taps and casks (who built that? - can someone spend a day and abstract away this useless distinction right in the brew code like I’ve done here :) - yeah and disallow a package identifier to be both while you’re at it ;) )
  • Benefits: broader reuse, declarative state approach (just “make it so”)

The Code

New versions of the code are not synced to the below article insert, please use the repository link below to get the latest.

function ifcmdmissing-instpkg () {
#If you don't need brew, this can be much more compact by removing the relevant code
#If command is not on path, installs the package
#detects package managers: apt, yum and brew
  if [[ -n "$(command -v brew)" ]] ; then
    #Detect package manager, brew first because some macOSes have an apt-get stub
    PKGMGR='brew'
  elif [[ -n "$(command -v yum)" ]] ; then
    PKGMGR='yum'
  elif [[ -n "$(command -v apt-get)" ]] ; then
    PKGMGR='apt-get'
  fi
  for cmdpkg in $1 ; do
      IFS=':' read -ra CMDPKGPAIR <<<"${cmdpkg}"
      CMDNAME=${CMDPKGPAIR[0]}
      PKGNAME=${CMDPKGPAIR[1]}
      echo "If command ${CMDNAME} is not on path, the package ${PKGNAME} will be installed."
      if [[ -n "$(command -v ${CMDNAME})" ]]; then
          echo "  '${CMDNAME}' command already present"
      else
          echo "  Missing command '${CMDNAME}'"
          echo "  Installing package '${PKGNAME}' to resolve missing command."
          if [[ $PKGMGR != 'brew' ]]; then
            if [[ $PKGMGR == 'apt-get' ]]; then $PKGMGR update; fi;
            $PKGMGR install -y ${PKGNAME}
          else
            #automatically abstract the difference between brew taps and casks
            if brew info ${PKGNAME} >/dev/null 2>&1; then
              brew install ${PKGNAME} --force
            elif brew cask info ${PKGNAME} >/dev/null 2>&1; then
              brew cask install ${PKGNAME} --force
            else
              echo "  '${PKGNAME}' not found as a formulae or cask"
            fi
          fi
      fi
  done
}

ifcmdmissing-instpkg "jq:jq wget:wget curl:curl"

Tested On

• MacOS
• Ubuntu container
• Amazon Linux 2 container

Source Code For This Article

ifcmdmissing-instpkg.sh

Appendix: Mission Impossible Pattern Philosophy

Mission Impossible Code samples are intended to be both 1) usable directly for production use and 2) a top notch pattern to use for your own innovation. More details on this approach are in the post

Appendix: Constructed Simplicity (Conciseness) is The Tip of An Iceberg

While epitomized by the Blaise Pascal quote “I have made this longer than usual because I have not had the time to make it shorter.”, I find that the process of creating concise, simple observations and solutions is complicated and sometimes complex.

Creating concise designs and solutions is a deep passion of mine. However, there is a frustration that is quick on the heals of creating something that is concise. In my line of work, at some point, you usually have to justify your final work. That is the point at which the rest of the iceberg of thinking that backs the concise tip comes to the fore. Frequently the reaction is that it can’t possibly be that involved or that you are spinning up reasons on the fly to simply bolster an idea or solution that was arrived at haphazardly.

Why bother addressing this sentiment? Because concise, mission impossible style, solutions can easily be criticized as lacking sophistication in their implementation. But much like a Mark Twain quote - that ruddy external appearance belies a hard wrought balance of many interrelated tradeoffs to get to a simple solution.

Said another way, solutions that are earnestly designed for conciseness can be rewarded by a perception of being the opposite of what they are - simplistic, backed with little or no thought.

This Mission Impossible Coding series exposes the submerged methodological icebergs below the waterline of the visible and concise solutions it attempts to arrive at.