Linking Dynamic Libraries with Executables in Swift

SHARE

Generally speaking, a library is a file contains compiled binary code, they are too useful as they provide the concept of reusability and allow for fast compilation time in modular applications. However, there are mainly two types of libraries, Static Libraries which are linked to the executable at compile time. Dynamic libraries at the other hand are linked to the executable at runtime. That makes them much more complicated and open the way for many possible hurdles which will be discussed in this article.

Oftentimes macOS loads all dynamic libraries (*.dylib) for an executable silently, but it sometimes happens to fail. To put that into a perspective, we will build XMLJson which is an executable file that has (dylib) dependencies if we build it with release configuration we will notice that it works perfectly inside the build folder, but what if we want to make it globally available? We can move the compiled binary to /usr/local/bin

$ cd .build/release
$ cp -f xmljson/usr/local/bin/xmljson

However, that’s will not work and we will get the following evil error:

dyld: Library not loaded: @rpath/libSwiftToolsSupport.dylib
Referenced from: /usr/local/bin/xmljson
Reason: image not found
[1] 44343 abort xmljson

The error seems pretty self-explanatory which tell us couldn’t find libSwiftToolsSupport.dylib library at [rpath] but we can break it down into pieces:

  • dyld: Is the macOS’s default dynamic linker tool.
  • couln’t find libSwiftToolsSupport.dylib at the specified runtime path (we will come to this later).
  • Image not found because the binary is missing @rpath.

Install Name

On macOS each dynamic library (dylib) has an "install name" property. The install name is a path baked into the dynamic library that says where to find the library at runtime. When you link against the dylib this path is saved in your binary (at link time) so that it can find the dylib later at runtime.

To solve our initial problem there are a set of options we need to demystify before going further:

  • @executable_path: This is a magic token that, when placed at the beginning of a library's install name, gets resolved to the absolute path of the executable that's loading it, minus the last component.

    Ex:  let's say that xmljson executable links against libSwiftToolsSupport.dylib . If xmljson is installed in: /usr/local/bin/, @executable_path will expand to : /usr/local/bin

  • @loader_path: It expands to the full path, minus the last component, of whatever is actually causing the target library to be loaded. For example, imagine xmljson depends on liba.dylib, and liba.dylib in its turn depends on libb.dylib, the path of liba will be resolved as the same for xmljson (the caller), and the path for libb will be resolved as the same path for liba (the caller).

  • @rpath: When placed at the front of an install name, this asks the dynamic linker to search a list of locations for the library. That list is embedded in the executable, and can therefore be controlled by the executable’s build process, not the library's. A single copy of a library can thus work for multiple purposes. The value of rpath can be set by the linker during linking the executable in two ways:

    1. In the Xcode, it's set with LD_RUNPATH_SEARCH_PATH setting.
    2. In ld command tool it's set with -rpath parameter when linking.

Executable Anatomy

To inspect the executable dependencies and their paths we have two handy command-line tools otool for mac and objdump for linux, run otool -L xmljson to list all its dependencies and their paths you will notice that all the dependencies are packed by absolute path so the executable knows where to load them except for our missing dependency is packed by runtime path.

XMLJson Dependencies and their paths

 

So far we have seen that the “install name” for “libSwiftToolsSupport.dylib” is @rpath/libSwiftToolsSupport.dylib that means it tells the dynamic linker wherever is the rpath I am there too. But to which value rpath in xmljson is set to? As we mentioned earlier @rpath can be set from the build settings in Xcode project by modifying LD_RUNPATH_SEARCH_PATH settings, let us inspect that too.

Clearly you can see that one of the rpath search paths is @executable_path , which explains why xmljson worked in the build folder but didn’t outside because both the executable and dynamic library living in the same directory.

So the solution is pretty easy, right? Just copy both the executable and dynamic library to the same directory i.e bin directory. However, this is a bad practice,  there is in fact lib directory dedicated for storing shared libraries. We will be good citizens and move our dynamic library to lib directory also change its install name to match the new path.

 

install_name_tool 

We have mentioned earlier that @rpath is part of the executable and is set during the linking time. However, if we want to change it later i.e. during the installation phase there is a handy command-line tool at our disposal: install_name_tool. It changes dynamic shared library install names, it has a bunch of commands but we interested in -change command, simply it changes the dependent shared library's old install name to a new one in the specified executable 

Here is the usage example:

 install_name_tool -change oldName newName executableFile

 

To apply this technique to xmljson executable:

  1. Copy xmljson from release folder to /usr/local/bin
  2. Copy libSwiftToolsSupport.dylib from release folder to /usr/local/lib
  3. Run the following command:

    install_name_tool -change \                                           ".build/release/libSwiftToolsSupport.dylib" \               "/usr/local/lib/libSwiftToolsSupport.dylib" \               "/usr/local/bin/xmljson"

What we are doing here is telling xmljson’s dynamic linker to change its old dependency path from build directory to the new path which is the lib directory. And it works again! There are many other handy options in install_name_tool you can find them here 

Conclusion

With all that said we have briefly covered what install name is and how to inspect binary objects and analyzing their dependencies. Hopefully now you have a little bit more about how  dynamic linker finds your libraries and how dynamic linking generally works  on macOS.

 

Please let me know your thoughts or any questions you might have on Twitter @alihilal94.

DYLDDynamic LinkingExecutableStatic Linking

You made it to the end. You're Awesome!

Here is something more to read

Any vs AnyObject in Swift: Everything You Need to Know

If you've spent some time writing Swift, then you've probably come across a variable, constant, or parameter of type AnyObject. And you know what AnyObject is. Right? Don't worry. You're not alone. Let's take a look and dive into the bowels of the Swift

From Open to Private: Swift Access Modifiers Explained

The access restriction to a specific piece of code is done through access controls. In Swift classes, structs, enums, and protocols can be accessed according to their modifiers. Luckily, it is very easy to learn in Swift and powerful at the same time.

This is a fully integrated open-source project that uses NextJS, Redux, and Django to build. Grab your copy from Github

Copyright © 2021 Ali Hilal. All rights reserved.