Keywords

1 Introduction

While there have been long-ranging debates on the most relevant programming language for Software Engineering as well as robotics [29, 31, 38], there is agreement on important characteristics that a programming language for robotics and embedded systems needs to satisfy. Number one is to be Turing completeFootnote 1. Importantly, it should be a systems programming language, able to provide fine grained, standardised control over devices, hardware drivers, and communication mechanisms, with predictable performance to enable integration across hardware, from sensors to actuators. The capability to have control over the temporal domain in execution, task control, and parallelism is crucially important. We argue here that such a language should also enable a seamless transition to higher behaviour-based descriptions and model-driven development.

We port Swift and its supporting environment to the virtual machine development environment associated with the SoftBank Pepper and Nao robots, as well as to the robots themselves. Swift has long produced fast and predictable executables for iOS and macOS, but is Open Source and features modern and safe language concepts. Complex data testbeds show Swift code to be about two times faster than Objective-C [39] and four to eight times faster than Python [1], while being similarly discoverable to programmers (e.g. through Playgrounds that facilitate interactivity during development and enable programmers to quickly test new algorithms). Swift’s modern features include closures, generics, and type inference, which have been shown to result in higher productivity and more adherence to common software patterns.

The clean syntax of Swift is a major factor for its higher readability [40] and thus maintainability, while its semantics make it harder to make mistakes common to systems programming languages and add a layer of quality control during development. Functional language features that provide referential transparency and avoid implicit state or mutable data allow writing performant code in more functional style [8]. This also facilitates pure functions and idempotence (i.e. lack of side-effects and the same output regardless how many times a function is called), enabling parallelisation as function calls become independent.

With Swift being Open Source, while being a relatively young language, and still somewhat of a moving target [33], it has become popular very quickly, including on the server side and cloud computing. The Swift language won first place for Most Loved Programming Language in the Stack Overflow Developer Survey 2015 and second place in 2016. Three years ago, Swift became the 12th most popular language, overtaking Objective-C, Go, Scala, and R. Now, in 2021, the reviews of the full-stack academy place Swift second just below JavaScript. For robotics, Swift provides the capability dynamic libraries and, vitally, a low memory footprint through value types and Automatic Reference Counting (ARC) that offers similar convenience as tracing garbage collectors, but vastly superior and deterministic performance. This is in contrast to languages such as Python, Java, C\(\sharp \), or Go, whose lack of ability to bound memory usage and garbage collection performance implies unpredictability of CPU usage, suggesting that every single thread on board of the robot could be denied vital CPU time, potentially resulting in catastrophic consequences. For example, what would be the use of a Kalman filter if the time-stamp of the sensor reading was unpredictable or seriously jeopardised? The literature has plenty of discussion [3] why, e.g., the versions of Java for real-time and/or embedded systems are not completely satisfactory.

While IDE integration is not as mature as with other languages and despite some blogs criticising Swift, suggesting that Objective-C can use C and C++ libraries more smoothly, we demonstrate that we can use in-memory middleware for module/package/node communication quite effectively, integrating applicable C and C++ libraries for robotics applications.

2 Cross Compiling

The goal of cross-compilation is to be able to build a program for a specific platform on a different platform. When compiling for the Nao and Pepper robots, SoftBank Robotics provides GNU tools commonly found in linux systems for C and C++ which have been specifically built to allow cross-compilation. The tools include binutils [15] which contains tools such as the linker (ld), glibc [16] which contains the C standard library, and the gcc project [14] which contains the C compiler (gcc) as well as other compilers such as the C++ compiler g++. This, for example, allows us to build Nao-robot applications from a 64-bit Linux or macOS. We note that the Nao and the Pepper use some version of the linux kernel (for us, we use version 2.6). The kernel is simply a C program, thus a C compiler is needed to compile the kernel.

When cross-compiling, two things are required on the host (the system executing the cross-compiler): the tools (the cross-compiler and linker), and, the sysroot folder containing all necessary files needed for compilation for the target platform, such as the Nao. The sysroot folder is usually setup to mimic the folder structure of the target and generally contains headers and libraries which are needed at runtime by the programs being compiled. When using the GNU tools (as Aldebaran did originally), a compiler must be provided for each host and target combinations. So in order to build for the Pepper and Nao robots on two separate hosts—for example macOS and linux—then you need a total of four compilers. Two compilers for each host which build for the Nao and Pepper respectively. However, we can use alternatives to this approach.

Our Swift project is built using LLVM [20], built on top of the GNU stack and provides compilers (clang for the C language and clang++ for C++) that allow programs to be cross-compiled without having to build a separate cross-compiler. In other words, the same compiler that is built for the host is able to cross-compile for other targets (as already utilised, e.g., for iOS and tvOS applications). We potentiate this further here for Swift, which already leverages LLVM, by creating an environment that allows us to cross-compile for Nao robots using only one cross-compiler.

3 Swift for the Pepper

To enable Swift for a new host system and target system, such as the Pepper, we need to build the Swift compiler (and libraries), often passing platform-specific flags to the C/C++ compilers to fix compiler errors or warnings. To ensure consistency, alleviate setup-specific issues, and document the process, we provide a Docker container which builds the Swift project [27]. Docker allows OS-level virtualisation in what is called containers, similar to virtualisation; however, rather than a host OS emulating the hardware in favour of a guest OS [37], containers share the host kernel. Containers are now a de-facto standard to share the environment for a package of applications.

Another challenge when building for the Pepper is the fact the Swift project does not support compiling for 32-bit linux targets. That is, none of the build scripts that are provided by the Swift project will allow such cross-compilation. Instead, we release our own scripts that our Docker container [27] automatically downloads and uses.

To cross-compile Swift for the Pepper, we need to replace existing binutils with gold enabled binutils (–enable-gold), build the LLVM tools for the host and the target, and then build the Swift standard library for the target. This allows us to create a swiftenv toolchain that enables building vital infrastructure libraries for the target such as libdispatch and Foundation.

3.1 Replacing Binutils and Building the LLVM Toolchain

SoftBank’s toolchain provides an older ld linker that does not have the capabilities required by more modern languages such as Swift. The Swift project requires the gold linker [41], ld.gold, a more modern alternative to ld. Our scripts download binutils and recompile them, enabling the gold linker and allowing it to be executed on the 64-bit host, but cross-compile for the 32-bit Pepper.

The Swift project uses a series of git repositories, utilising a consistent version tag, to distribute a specific version of the language and the tools that are used within the Swift toolchain. The version of Swift that we have built is version 5.1.4 (tagged as swift-5.1.4-RELEASE).

We then build the LLVM project, creating C/C++ compilers that are capable of running on the host, but also cross-compile for the Pepper. We recommend building the C/C++ compilers using our Swift project since subsequent cross-compilation of robotic applications with Swift is simpler and more convenient. In particular, fewer flags are necessary for cross-compilation than with the GNU cross-compilers provided by the original SoftBank toolchain.

Using the host tools built in the previous step, a second build cross-compiles the LLVM toolchain for the Pepper delivering pre-requisites for the Swift cross-compiler and standard library. This requires several flags that inform the host C/C++ compiler where the shared objects, headers and C standard library exist within the Pepper toolchain. This is necessary to redirect the compiler away from the usual locations in the 64-bit host environment.

3.2 Building the Swift Environment for the Pepper

As we cross compile, we can skip building a Swift compiler to run on the Pepper, but, the Swift standard library must still be built, which is now possible with our cross-compilers. However, The Swift standard library consists of several modules written in different languages, some are C/C++ while others (such as the main standard library) are written in Swift itself and needs to be compiled and installed into the sysroot. Importantly here, since the standard library is written in Swift, a Swift compiler with a specific version is needed within the Docker container. We use swiftenv [13] in order to download the appropriate compiler for the version of Swift that we are compiling, to ensure that the host Swift compiler and all the other tools that exist within the toolchain match the versions of the tools that we are building for the Pepper.

The Swift project follows a folder structure where the actual Swift standard library folders are placed within a Swift folder under the installation directories lib folder (typically /usr/lib for a normal installation). The installation folder that our scripts use is $SYSROOT/home/nao/swift-tc where $SYSROOT represents the absolute path of the sysroot directory on the build system. The reason why we use this installation prefix is to follow Pepper’s security policy where the only folder with write access is the /home/nao folder. This swift directory follows the following format:

figure a

In this folder structure \(\texttt {{<}os{>}}\) and \(\texttt {{<}arch{>}}\) represent the target operating system and architecture (e.g. linux and i686 for the Pepper). These folders contain the Swift standard library files, consisting of the shared objects or dynamic libraries required by the compiler. The glibc.modulemap file includes the C standard library which allows developers to import into their programs by a simple import statement, for example (import Glibc).

This folder structure is important because when building the Pepper Swift toolchain, some of these files need to change. Recall that we leverage swiftenv to install a version of Swift that matches the version of Swift that we are building for the host. This makes it so that we can use the host Swift compiler to cross-compile Swift programs. However, the host Swift toolchain contains the standard library for the host 64-bit linux. We copy this toolchain into a new Pepper toolchain folder, and replace the Swift folder contents with the cross-compiled toolchain that was installed into the sysroot, replacing the 64-bit standard library with the 32-bit standard library built for the Pepper. Then we patch the glibc.modulemap file to correct the hardcoded paths to point to the Pepper C standard library within sysroot.

3.3 Building Libdispatch and Foundation for the Pepper

Once the Pepper swiftenv toolchain has been created, we can use it to build the remaining projects that make up the required Swift infrastructure. The libdispatch project is a C project, but the Swift variant contains wrappers that enable simple access to libdispatch from Swift programs. The Foundation framework was once written in Objective-C, but has since been rewritten in pure Swift. This allows it to be deployed to linux platforms, adding extra functionality on top of the Swift standard library. By using the swiftenv, we are able to compile these components and install them within the sysroot.

4 Swift on Legacy Robots

Older versions of the Nao (version 5 or earlier) use older versions of the C compiler and standard library that fail to meet the minimum system requirements for cross-compiling the swift toolchain. To overcome this issue, we need to take the extra step of installing an entirely separate root directory within the home directory of the Nao that will contain all files required for a newer linux system containing a suitable version of glibc. However, we avoid installing an entire linux system. Instead, we install the minimum amount of software that Swift needs in order to minimise disk space.

Installing a newer version of glibc is particularly challenging, it is an integral part of the system, i.e., there is a mutual inter-dependence between the kernel and the GNU C compiler (gcc) on the one hand and glibc on the other hand. This circular dependency creates close coupling, and if the newer version of glibc breaks ABI (application binary interface) stability (meaning that symbols within the library have changed), programs that have already been compiled will stop working. Since the linux kernel is itself a C program which links against glibc, this means that nothing will work. Upgrading an existing glibc to a newer version is therefore only possible if the new glibc version does not break ABI stability. In our illustration (Nao V5), the minimum version of glibc supported by the swift toolchain does break ABI stability.

An additional challenge is that the glibc version that is needed for the swift toolchain does not compile under the Nao kernel. For our approach of a parallel system root, the new toolchain must not depend on anything outside of this parallel system root. If we were to simply compile glibc using the existing gcc compiler, then the new glibc would link against libgcc.so which is outside of the system root directory. Hence, we needed to compile an entirely new linux systems from source, following [2]. This book provides a comprehensive guide for creating new Linux distributions without any prior knowledge, particularly, the details for compiling a self-contained system root [2]. Akin to this approach, we created a parallel system root containing the new glibc, kernel and gcc, taking the opportunity to upgrade the gcc compiler to a more modern version. From this point, the same steps as in Sect. 3 apply.

5 Efficient Robot Software Architectures Through Swift

5.1 Behaviour-Based Architectures

Finite-State Machines are ubiquitous models of behaviour. In robotics, they have been used in several forms, perhaps most notably with the introduction of the subsumption architecture [4] and therefore, in behaviour-based control by the seminal description of Toto [22,23,24]. Finite-state machines are typically associated with reactive software architectures for robotics, because of the prevalence of the state-chart format introduced by Harel [18]. However, Harel’s semantics of finite-state machines is event-driven, implying that the driver of any activity is the environment who generates events to state-charts waiting and expecting events [7]. Harel’s vision was the creation of mechanisms to compose more sophisticated behaviours by hierarchies of state-charts as nesting one sub-machine in a parent sub-machine created an appealing mechanism nesting behaviours from other behaviours. This idea gained acceptance in OMT [35] and latter became the dominant semantics of UML’s model for describing behaviour [32]: that is the semantics of run until completion [7, 36].

However, the original time-augmented finite-state machines of the subsumption architecture were essentially logic-labelled; that is, what labels a transition between two states is a Boolean expression. In such models, the machine runs at its own pace, evaluating the expressions of the transitions and performing a transition only when the expression evaluates to true. Expressions labelling transitions can be as sophisticate as decision trees [21, 34]. Logic-Labelled Finite State Machines (LLFSMs) use this alternative paradigm of execution enabling much safer elaboration of behaviours [30]. Rather than using nesting, LLFSMs can use stacking like the subsumption architecture [4] or create versatile dynamic executions by suspend, resume and restart calls. Concurrency of arrangements of LLFSMs is achieved by a sequential pre-defined schedule of execution, enabling smaller state-spaces for formal verification and model checking [12].

5.2 Formal Verification

Implementations of LLFSMs were based on interpreters until clfsm [12] offered their compilation into loadable libraries in C++. However, doing so left them unverifiable as the C++ language does not posses any means of being able to query the state of the variables making up the LLFSM at run time. Our porting of Swift to the SoftBank robots enables efficient execution on board of the robot of compiled swift LLFSMs while still enabling formal verification. Here we argue for the use of swiftfsm [25, 26]: a scheduler for LLFSMs enabling formal verification.

Fig. 1.
figure 1

The temperature monitor LLFSM

Full size image

We now present a small illustration of the efficacy and usefulness of our Swift toolchain. We use swiftfsm to perform verification of a small piece of infrastructure: a module which monitors the temperature levels of the motors on the robot. The corresponding LLFSM is presented in Fig. 1. For the sake of this example, we represent the temperature sensors as Boolean values. This is to minimise the resulting Kripke Structure from the verification for this simple example. Each Boolean variable in the external variables represents whether or not that particular sensors is overheating. If the sensor is overheating, then the value is true, otherwise the value is false. The overheating external variable represents a control message which notifies any other modules that a joint is overheating. Thus, this LLFSM simplifies the process of checking whether the robot is overheating by distilling the sensors into one overheating message.

The machine starts in the Monitor state indicated by the initial pseudo-state icon. The Monitor sets overheating to false and contains a number of transitions to the Overheating state. We have labelled each of these transitions with a check for each external variable. Therefore, if any of these external variables evaluate to true, then the machine will transition to the Overheating state. Our machine takes advantage of the LLFSM semantics [10] implemented by swiftfsm to continuously poll the external variables throughout its execution. This simple fact also demonstrates the advantages of swiftfsm in handling issues of concurrency without any overhead created by the user. Lastly, if the machine transitions to the Overheating state, then the overheating external variable is set to true. After a second and when all sensors are not overheating, the machine transitions back to the Monitor state thus setting overheating back to false.

One of the advantages of using Swift and swiftfsm is that swiftfsm utilises an extensive set of protocols to enforce its semantics. A protocol in Swift is a construct similar to an interface in Java, but more powerful. This protocol oriented design (a paradigm widely used in the Swift community) establishes the desired semantics of the scheduler which enables formal verification. Only by enforcing LLFSMs to follow these semantics—through the use of protocols—is a verification possible [28]. Performing the actual verification is only possible because of the reflection capabilities of the language. As stated earlier, the C++ variants of LLFSMs were only able to verify LLFSMs through the implementation of an interpreter. With Swift, we are able to verify LLFSMs natively without having to implement a custom interpreter. Not only does the temperature monitor LLFSM demonstrate these features of Swift, it also demonstrates one of the necessary features for robotics: interoperability with C.

In the temperature monitor, each external variable maps to a message within a middleware implemented in C: the gusimplewhiteboard [12]. Through the use of this middleware, any Swift LLFSM (and by extension any Swift program) is able to communicate with any other modules within the system. Many languages support C bindings making the use of a middleware a key step in being able to communicate between modules written in different languages including C++. The temperature monitor demonstrates that the use of Swift does not demand that existing modules must be rewritten. Swift modules can add extra functionality and coexist with existing modules written in different languages. Other LLFSMs written in C++ or Swift can now use the output of the Swift machine in Fig. 1 through the use of the same middleware. Moreover, using LLFSMs enables Model-Driven Software Development. That is, the LLFSMs can be defined using a language agnostic meta-model and editor. Then, model-to-text transformation produces the equivalent behaviour for C++, Swift, and LISP [5]. More importantly for formal verification, a model-to-text transformation produces the equivalent NuSMV model (see coming up example in Fig. 3b) which can be co-simulated by the model-checker (that is, by NuSMV) as well as verifying properties against it.

5.3 Deliberative Architectures

Importantly, LLFSMs enable the ability to create declarative and deliberative architectures, as the expression labelling the transitions can be a query to a reasoning agent [11] or a task planning system [9].

We now illustrate LLFSMs capability to describe behaviours beyond a reactive architecture. Our example is part of an application of a social robot that plays the game of Spanish dominoes with a human partner against another pair of players [19]. Figure 2 shows a small Prolog program that defines whether a player can play one of the tiles in their hand or must pass. Figure 3a shows an executable (compiled, not interpreted) of deliberative LLFSMs of a player that plays with the most naive strategy, play the first tile playable in the hand or pass. A more sophisticated player can be produced by sophistication of the Prolog program of Fig. 2. Nevertheless, the current version is sufficient to implement the server of the game that monitors players following the rules to play valid tiles and not revoke (claim to pass when it is possible to play).

Fig. 2.
figure 2

Determine a tile to play in the players hand.

Full size image

The experience of enabling Swift is crucial to port GNU-Prolog [6] to the robots. Porting the interpreter gprolog and the compiler gplc for interfacing with C amounts to downloading and patching the sources of a suitable version and using out C/C++ cross-compilers. However, calling prolog from Swift LLFSMs requires cross-compiling a C/C++ wrapper. In our illustration we wrap the querying whether a player holding a specific hand can lay a tile on the board and must pass. This wrapper is now a C-function that is in itself wrapped and linked by Swift and thus, we can invoked in the label of a transition of the LLFSM defining the player’s behaviour. However, this is easier said than done. While our earlier infrastructure enables to install gprolog on the host (the docker container) and to cross-compile gprolog for the Pepper, the gplc compiler offers no options to specify target architectures. Thus, although we can invoke our C cross-compiler for the .c files in the prolog project, we need to compile the .pl (the pure Prolog files) on the Pepper.s assembly files:

  1. 1.

    pl2wam -o step.wam step.pl

  2. 2.

    wam2ma -o step.ma step.wam

  3. 3.

    ma2asm -o step.s step.ma

We then copy the .s files back to the Docker host and continue with the cross compilation of the .s files to .o (object) files using the Pepper cross-assembly compiler. Now all files are linked in the host using our cross-linker and we have an executable (binary) that once copied to the Pepper executes as expected.

Fig. 3.
figure 3

Small example of an LLFSMs that enables deliberation. LLFSMs use a cross-action language semantics that enables automatic translation to NuSMV.

Full size image

6 Conclusions

Model-driven software development promises high levels of abstraction. In this paper we have shown how to incorporate a modern systems programming language, Swift, with the most advanced features of object orientation that enables complete execution time control. We incorporated features of functional and logic programming. All this enables a high-level of abstraction that define executable behaviour models. Moreover, we can automatically produce input code for a model-checker for formal verification. We believe that these paradigms significantly improve the quality of the robotic behaviours and the facility to produce correct code both in the value domain as well as the time domain. With the combination of paradigms, we are not advocating just a choice of programming language, we are enabling safety through adherence to widely accepted design principles and philosophies. This is more important than the specific language; however, we believe the experience reported in this paper for enabling Swift as well as our Docker container constitute immediate, valuable contributions and pieces of knowledge for broadening the available tools in the RoboCup community.