YDPA #02: You Don't Pay to do #Acoustics

JayJay
6 min read

(cover credit @askubuntumemes)

We will be finally setting our foot on the ground! Feel free to clone the repository to follow along:

What you need before starting

First of all, you need the common developer tools available on Mac: git, gcc, make and g++. If they aren't, proceed to Terminal and run through xcode-select --install.

And, make sure you have Homebrew installed!

Finally, we need a few packages from Homebrew before proceeding, they are essential for building the dependencies needed in later stages:

brew install pcre automake

Dependencies

A glimpse of the CI configurations at appveyor.yml and ci/travis/linux.yml for (Appveyor and Travis CI respectively) reveals the general flow of building the software:

  1. Download source codes of third-party packages

  2. Install the packages with custom options (using make)

  3. Navigate to the project directory, compile

  4. Pack, install, and test the compiled artifacts

The main packages used are:

  • python 3.8 : Needed for compiling, in case of user-supplied scripts

  • boost 1.73.0 : Used in EVERY module

    • Sublibraries: filesystem,system,test,regex,python,random,thread,timer,date_time
  • swig 3.0.10 : Transforming numerical codes into python external packages, which could be used by user-supplied scripts

  • wxWidgets 3.1.4 : Rendering interactive components in the User Interface

  • CMake 3.17.2: For building the software

  • OpenGL (in a subtle way): Rendering 3D models in the User Interface

โ˜น
While using third-party package managers (conan, vcpkg) could be beneficiary in the long term, this goal is suspended at the moment due to compatibility concerns.

Let's have fun :)

Boostrapping

In case the version of python does no match exactly on your machine (it was shipped in xcode-select --install), you can install it via brew install python@3.8 or manually compiling the source code.

> python3 --version
Python 3.9.6

In the former case, brew linked the binary to python3.8 automatically.

> which python3.8
/usr/local/bin/python3.8

Now let's follow along with the Linux build scripts from the original repository:

# ci/travis/linux/before_install.sh
#
# Boost install

if [ -d $HOME/boost-install/boost ] ; then
    echo "Boost already built (and in travis cache)"
else
    cd
    wget https://sourceforge.net/projects/boost/files/boost/1.73.0/boost_1_73_0.tar.bz2
    tar -xjf boost_1_73_0.tar.bz2 --strip-components=1 -C $HOME/boost-install
    ls -l $HOME/boost-install
    cd $HOME/boost-install && echo "using python : 3.8 : /usr/bin/python3 : /usr/include/python3.8 : /usr/lib ;" > tools/build/src/user-config.jam
    cd $HOME/boost-install && ./bootstrap.sh link=static variant=release address-model=64 cxxflags="-std=c++11 -fPIC" boost.locale.icu=off --with-libraries=filesystem,system,test,regex,python,random,thread,timer,date_time --prefix=$HOME/boost-install && ./b2 install
fi
export BOOST_LIBRARYDIR=$HOME/boost-install/lib/
export BOOST_INCLUDEDIR=$HOME/boost-install/
export BOOST_ROOT=$HOME/boost-install/

#
# Swig install
if [ -f $HOME/swig-install/bin/swig ] ; then
    echo "Swig already built (and in travis cache)"
else
    cd
    wget https://github.com/swig/swig/archive/rel-3.0.10.tar.gz
    tar zxvf rel-3.0.10.tar.gz
    mkdir $HOME/swig-install
    cd  $HOME/swig-rel-3.0.10 && ./autogen.sh && ./configure --prefix=$HOME/swig-install && make && make install
fi

#
# WXWidget install

if [ -d $HOME/wxWidgets-install/include ] ; then
    echo "wxWidget already built (and in travis cache)"
else
    cd
    wget https://github.com/wxWidgets/wxWidgets/releases/download/v3.1.4/wxWidgets-3.1.4.tar.bz2
    tar -xjf wxWidgets-3.1.4.tar.bz2
    mkdir $HOME/wxWidgets-install
    cd $HOME/wxWidgets-3.1.4 && ./configure --prefix=$HOME/wxWidgets-install --disable-shared && make && make install
fi

The above steps could take a while, it is straightforward enough that the packages are downloaded, and built under designated locations. These locations would be referenced by CMAKE in later stages.

๐Ÿ’ก
In case wget https://x.y.z is not available, use curl -LOJ https://x.y.z instead. Pay attention to the names of downloaded files.

And finally, we download CMake and make sure everything is okay.

Note the change of file name from cmake-3.17.2-Linux-x86_64.tar.gz to cmake-3.17.2-Darwin-x86_64.tar.gz, and the strip value.

# check wxWidget install
export PATH=$HOME/wxWidgets-install/bin/:$PATH
echo "wxWidget version : "
wx-config --version

# Download CMake
cd
wget --no-check-certificate curl -LOJ https://cmake.org/files/v3.17/cmake-3.17.2-Darwin-x86_64.tar.gz
mkdir $HOME/cmake-install
tar zxvf cmake-3.17.2-Darwin-x86_64.tar.gz -C $HOME/cmake-install --strip 3

Compiling with CMake

librt and clock_gettime()

# ci/travis/linux/install.sh
mkdir build
cd build

CLANG_WARNINGS=""

# wxWidget path (only in script scope)
export PATH=$HOME/wxWidgets-install/bin/:$HOME/swig-install/bin/:$HOME/cmake-install/bin/:$PATH
export BOOST_LIBRARYDIR=$HOME/boost-install/lib/
export BOOST_INCLUDEDIR=$HOME/boost-install/
export BOOST_ROOT=$HOME/boost-install/

cmake --version
# ${CC} --version
# ${CXX} --version

ls $HOME/boost-install/lib/

cmake ..

Here the original script is fully applicable to MacOS environments, theoretically (cough). Now CMake comes into play.

-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - not found
CMake Error at src/spps/CMakeLists.txt:74 (message):
  clock_gettime not found
-- Configuring incomplete, errors occurred!

And at src/spps/CMakeLists.txt:

# Check for clock_gettime function
if (UNIX)
  include(CheckLibraryExists)
  check_library_exists(rt clock_gettime "time.h" HAVE_CLOCK_GETTIME )
  if(NOT HAVE_CLOCK_GETTIME)
    message(FATAL_ERROR "clock_gettime not found")
  endif(NOT HAVE_CLOCK_GETTIME)
endif(UNIX)

The issue is, although OSX is Unix, but neither clock_gettime() nor librt is implemented. We could get away with this error simply by narrowing the if-condition.

-if (UNIX)
+if (UNIX AND NOT (${CMAKE_SYSTEM_NAME} MATCHES "Darwin"))
   include(CheckLibraryExists)
   check_library_exists(rt clock_gettime "time.h" HAVE_CLOCK_GETTIME )
   if(NOT HAVE_CLOCK_GETTIME)
     message(FATAL_ERROR "clock_gettime not found")
   endif(NOT HAVE_CLOCK_GETTIME)
-endif(UNIX)
+endif(UNIX AND NOT (${CMAKE_SYSTEM_NAME} MATCHES "Darwin"))

And vice versa for another snippet in the same file:

-  if(UNIX)
+  if(UNIX AND NOT (${CMAKE_SYSTEM_NAME} MATCHES "Darwin"))
     target_link_libraries (spps rt)
-  endif(UNIX)
+  endif()

Now you should be able to generate the build files under build/ .


OpenGL Tweaks

Remember what the repository said? Some special CMake directives of OpenGL are needed. Let us try running the build script first: make VERBOSE=1:

src/isimpa/./GL/opengl_inc.h:40:10: fatal error: 'GL/gl.h' file not found
#include <GL/gl.h>
         ^~~~~~~~~
1 error generated.

Hence, the OpenGL header is not properly included. A solution taken from StackOverflow hints the correct way:

#if defined(__APPLE__)
#include <OpenGL/gl.h>
#include <OpenGL/glu.h>
#else
#include <GL/gl.h>
#include <GL/glu.h>
#endif

Make the appropriate changes to all the files containing similar headers (except the ones specifically for Windows), run make install, then lastly you will be able to see the precious binary file at build/bin/isimpa !


๐Ÿ˜› Sad news!

> ./bin/isimpa
dyld[4253]: Library not loaded: @rpath/libboost_system.dylib
  Referenced from: <C9FD3A5A-FDBC-3356-ABCF-5CBF7085B819> /Users/asuwish/Documents/sandbox/J-Simpa/build/bin/isimpa
  Reason: no LC_RPATH's found
zsh: abort      bin/isimpa

Why is that? We are indeed linking to the dynamic libraries of boost (like .dll as in Windows, .dylib as in Mac) when we run the executable every time. And we did not supply LC_RPATH at build time, so the PC has no way to find out where the library is located. By using otool (shipped in OSX), we can identify the loading path of respective binaries:

> otool -L ./bin/isimpa
./bin/isimpa:
        @rpath/libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
        @rpath/libboost_python38.dylib (compatibility version 0.0.0, current version 0.0.0)
        @rpath/libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
        @rpath/libboost_regex.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/Python (compatibility version 3.8.0, current version 3.8.0)
        ...

While a compact fix requires further effort, here is a simple fix:

> install_name_tool -change @rpath/libboost_system.dylib $HOME/boost-install/lib ./bin/isimpa
> ./bin/isimpa:
        /Users/abc/boost-install/lib/libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0) (compatibility version 0.0.0, current version 0.0.0)
        ...

After replacing all the rpath... Voila!

๐Ÿ’ก
There are obviously smarter ways to optimize. We will try it out on Part 3! See you :)
0
Subscribe to my newsletter

Read articles from Jay directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Jay
Jay

An acoustician, a DevOps engineer and a pen-tester :)