ArmarX Setup Tool

Setup and maintenance tool for ArmarX and its dependencies.

Installation

If not done yet, install these system packages:

sudo apt install git gcc python3-venv libpython3-dev

libpython3-dev is required by python-Levenshtein which is used to suggest alternatives on typos.

Clone the repository, e.g. into ~/armarx-setup:

cd ~  # Or another directory of your choice.
git clone git@gitlab.com:ArmarX/meta/setup.git armarx-setup
cd armarx-setup/

Run bootstraph.sh. This sets up a virtual python environment .venv and makes the armarx-setup command globally available:

./bootstrap.sh  

Pass --no-bashrc if you do not want to have your ~/.bashrc changed automatically.

Open a new console or source the ~/.bashrc by running:

source ~/.bashrc

Optional: Enable shell completion (needs to be done only once):

armarx-setup self enable-completion

To update the setup tool itself, run:

armarx-setup self update

Getting Started

To get going, initialize a new workspace:

armarx-setup init

Unless you know what you do, choose yes when you are asked about modifying the ~/.bashrc. (See Directory-Aware Environment below for how to use multiple workspaces.) Afterwards, open a new console.

To run the setup, upgrade your workspace:

armarx-setup upgrade

By default, this will install the ArmarX framework packages and their dependencies.

For more details, see Usage below.

Usage

See which commands are available:

armarx-setup --help

Create a Workspace

Initialize a new workspace:

armarx-setup init

The active workspace is specified by the ARMARX_WORKSPACE environment variable. It can be set by sourcing the file armarx-workspace.rc generated in your workspace. If you chose to source the created armarx-workspace.rc in your ~/.bashrc during the workspace initialization (recommended), open a new console or run source ~/.bashrc for this to take effect.

If you want to use multiple workspaces instead (i.e. not have ARMARX_WORKSPACE set to a specific workspace by default), have a look at the section Directory-Aware Environment below.

Manage a Workspace

Introducing Modules and the Module Graph

The ArmarX Setup Tool is based on modules.

Let's start with an example:

• There is a module armarx/ArmarXCore which clones, configures, builds and installs the ArmarX package ArmarXCore.
• In addition, the module armarx/ArmarXCore depends, or requires, another module named simox, which clones, configures, builds and installs the standalone CMake package Simox.
• The module simox, in turn, requires the module apt/eigen3, which installs the linear algebra library Eigen via a system package.

This dependency chain can be represented by a directed (acyclic) graph, where modules are nodes and dependencies build edges,

(armarx/ArmarXCore) --> (simox) --> (apt/eigen3)

or, visually:

This module graph is the heart of ArmarX Setup. In general, modules can clone repositories, build projects, install system packages, and more. Most importantly, modules have dependencies to other modules (their required modules). They build the module graph, which represents a program of how your workspace should be set up.

Apropos: Your workspace specifies one or several modules it would to have installed in its config file armarx-workspace.json. These modules are added to the module graph, as well as their dependencies, and their dependencies, and so on.

You can even visualize the active module graph by running (optional arguments in []):

armarx-setup info --graph [--reduce] [--with-system]

By the way, the module name, e.g. armarx/ArmarXCore, is also the path inside your workspace where a module representing a code base will be cloned or downloaded to (e.g. ~/code/armarx/ArmarXCore if your workspace is ~/code). At the same time, they are paths to .json files in the data/modules directory, which specifies what the module does.

Managing Workspace Modules

You can modify the workspace by adding or removing modules:

armarx-setup add module/name
armarx-setup remove module/name

# Examples:
armarx-setup add armarx/ArmarXCore
armarx-setup add armarx_integration/robots/armar6

To list the modules added to the workspace:

# List just the added modules:
armarx-setup list

# List the added modules and their direct and indirect dependencies.
armarx-setup list --deps

# List all modules in the database (independent of the workspace):
armarx-setup list --all

To search for a module, you can do:

armarx-setup list --all | grep -i "search term"

To show detailed information about the current workspace:

armarx-setup info

# To include pure system modules:
armarx-setup info --system

# To plot the module graph (requires additional packages, see setup below):
armarx-setup info --graph
armarx-setup info --graph --reduce  # See the transitively reduced module graph

Upgrade a Workspace

The most important command is upgrade. Use it to

• install system requirements such as apt packages (you are asked beforehand),
• update/clone repositories,
• prepare (e.g., run cmake),
• build (e.g., run cmake --build), and
• install

all modules and their (direct and indirect) dependencies:

armarx-setup upgrade

You can skip any of these steps, for example (see armarx-setup upgrade --help):

armarx-setup upgrade --no-system --no-update

You can also directly run one of these steps:

armarx-setup system
armarx-setup update  # add '--parallel' to speed this up
armarx-setup prepare
armarx-setup build
armarx-setup install

You can also execute a custom command in each module's directory inside the workspace (if it exists):

armarx-setup exec ls
armarx-setup exec "echo \$(pwd)"

The Local Workspace Config (armarx-workspace.json)

The local workspace config file (armarx-workspace.json in your workspace directory) contains the added modules as well as global module settings that will be used by all modules (if appropriate):

{
  "modules": {
    ...
  },

  "global": {
    ...
  }
}

At the moment, you can specify

• CMake definitions during the prepare step (prepare/cmake/definitions/)
• The CMake generator (prepare/cmake/generator/)
• Environment variables during the build step (prepare/cmake/env)

If you need additonal settings, please create an issue.

{
  "global": {
    "prepare": {
      "cmake": {
        "definitions": {
          "KEY": "VALUE",
          ...
        },
        "generator": {
          "name": "Ninja" / "Unix Makefiles",
          "args": "..."
        }
      }
    },
    "build": {
      "cmake": {
        "env": {
          "KEY": "VALUE",
          ...
        }
      }
    }
  }
}

Parallel Build

To build with multiple cores, add this to your local workspace config (replace 7 by the number of cores appropriate for your system):

{
  "global": {
    "build": {
      "cmake": {
        "env": {
          "CMAKE_BUILD_PARALLEL_LEVEL": "7"
        }
      }
    }
  }
}

Disable tests (ArmarX projects only)

To disable tests, add this to your local workspace config

{
  "global": {
    "prepare": {
      "cmake": {
        "definitions": {
          "ARMARX_BUILD_TESTS": "OFF"
        }
      }
    }
  }
}

Disable CMake's global registry

Some projects such as all ArmarX packages might register themselves in the global CMake registry found in ~/.cmake/packages. This however violates the concept of isolated workspaces. All modules should expose the CMake project they contain by providing a project_DIR environment variable. Therefore, it is advised to disable the global registration via the following

{
  "global": {
    "prepare": {
      "cmake": {
        "definitions": {
          "CMAKE_EXPORT_NO_PACKAGE_REGISTRY": "ON"
        }
      }
    }
  }
}

Use Ninja instead of Unix Makefiles

To use Ninja as your default CMake generator, simply adjust your global workspace config like so:

{
  "global": {
    "prepare": {
      "cmake": {
        "generator": {
          "name": "Ninja",
          "args": "-k9999"
        }
      }
    }
  }
}

Make sure that the apt package "ninja-build" is installed.

Ninja only: colored output

In contrast to make, the compiler output won't contain any colors if ninja is used. To overcome this limitation, add the following to the workspace config

{
  "global": {
    "prepare": {
      "cmake": {
        "definitions": {
          "FORCE_COLORED_OUTPUT": "ON",
          "ARMARX_CMAKE_ADDON_FORCE_COLORED_OUTPUT": "ON"
        }
      }
    }
  }
}

Use Ccache

To make use of Ccache, it is advised to depend on the apt/ccache module to export the $CCACHE variable, and then set the corresponding CMake variables in the global workspace config as follows:

{
  "modules": {
    "apt/ccache": {},
    ...
  },

  "global": {
    "prepare": {
      "cmake": {
        "definitions": {
          "CMAKE_C_COMPILER_LAUNCHER": "$CCACHE",
          "CMAKE_CXX_COMPILER_LAUNCHER": "$CCACHE"
        }
      }
    }
  }
}

(Optional) Define the cache size (example, 100GB) (after upgrading once to install Ccache):

ccache -M 100G

IDE integration

VS Code

Define the build tasks in ~/.config/Code/User/tasks.json. The following will build the project belonging to the currently opened file:

{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "ax_build",
      "type": "shell",
      "command": [
        "armarx-setup build -n -f",
        "${file}"
      ],
      "problemMatcher": {
        "owner": "cpp",
        "fileLocation": [
          "relative",
          "${workspaceRoot}/build"
        ],
        "pattern": {
          "regexp": "^(.*):(\\d+):(\\d+):\\s+(warning|error):\\s+(.*)$",
          "file": 1,
          "line": 2,
          "column": 3,
          "severity": 4,
          "message": 5
        }
      },
      "group": {
        "kind": "build",
        "isDefault": true
      }
    }
  ]
}

You can also update your keybindings (~/.config/Code/User/keybindings.json)

{
      "key": "ctrl+b",
      "command": "workbench.action.tasks.build"
}

Then just press Ctrl+B to trigger the build.

Advanced Usage

Instead of running bootstrap.sh, you can also manually set up the tool's environment.

Create the virtual python environment by either (1) or (2):

# (1) Using pip and virtalenv
python3 -m venv .venv
source .venv/bin/activate
pip install wheel  # This must be done before requirements.txt due to python-levenshtein.
pip install -r requirements.txt

# (2) Using Python Poetry
poetry install

Optional: To allow plotting the module graph, install:

sudo apt-get install graphviz graphviz-dev
pip install -r plotting_requirements.txt

./bin/armarx-setup is a bash script which first activates the python environment, then calls the python script ./armarx-setup in the root directory if this repository (.).

If you have activated the python environment, you can also directly call ./armarx-setup or python -m armarx_setup.

To be able to run armarx-setup from anywhere, add this line

export PATH=$HOME/armarx-setup/bin:$PATH

to your ~/.bashrc by running (take care to use 'single quotes', not "double quotes"):

echo 'export PATH=$HOME/armarx-setup/bin:$PATH' >> $HOME/.bashrc

Then source the ~/.bashrc or open a new console.

Directory-Aware Environment (optional)

Instead of sourcing the armarx-workspace.rc file in your .bashrc, armarx-workspace.rc can also be sourced when entering the workspace in the terminal. To do so, you need to install direnv first

sudo apt install direnv

Depending on the shell you use, you need add the following hook to your ~/.bashrc (see https://direnv.net/docs/hook.html for more details).

eval "$(direnv hook bash)"

# suppresses the output when entering the directory
export DIRENV_LOG_FORMAT=

Open a new terminal. Within the root of your workspace, create a file .envrc with the following content

source armarx-workspace.rc

Save and close the file. There will be a warning, that you need to execute

direnv allow

If you do so, this file will always be sourced when you enter the directory.

Aliases

If you would like to use an alias command instead of armarx-setup and want to keep the shell completion, you can do it like this (e.g. for the alias axii):

cd .../armarx-setup/bin/
ln -s armarx-setup axii

To enable shell completion for the alias, open the file armarx-setup/armarx_setup/cli/config_files/armarx-setup-complete.bash and change

...
_armarx_setup_completion_setup() {
    complete -o nosort -F _armarx_setup_completion armarx-setup
}
...

to (replacing axii with your alias):

...
_armarx_setup_completion_setup() {
    complete -o nosort -F _armarx_setup_completion armarx-setup
    complete -o nosort -F _armarx_setup_completion axii
}
...

You can also define custom aliases for special calls by adding the following lines to your ~/.bashrc:

# build the project in this directory and its dependencies
alias bt="armarx-setup build -t"

# build the project in this directory without its dependencies
alias btn="armarx-setup build -t -n"

# build all projects and ignore errors
alias ba="armarx-setup build -i"