Quick setup guide¶

Before going further, make sure you have ROS 2 (jazzy or humble) installed and sourced on your system.

Docker images

RAI has experimental docker images. See the docker for instructions.

There are two ways to start using RAI:

Installing RAI using pip (recommended for end users)
Setting up a developer environment using poetry (recommended for developers)

Installing RAI¶

Virtual environment

We recommend installing RAI in a virtual environment (e.g., virtualenv, uv, or poetry) to keep your dependencies organized. Make sure to use the same version of python as the one used for ROS 2 (typically python3.10 for Humble and python3.12 for Jazzy).

If you plan to use ROS 2 commands (ros2 run or ros2 launch), you'll need to add your virtual environment's Python packages to your $PYTHONPATH. This step is only necessary for ROS 2 integration - if you're just running RAI directly with Python, you can skip this step.

For reference, here's how to set this up when installing RAI from source: setup_shell.sh

Install core functionality:
```
pip install rai-core
```
Initialize the global configuration file:
```
rai-config-init
```

Optionally install ROS 2 dependencies:

sudo apt install ros-${ROS_DISTRO}-rai-interfaces

Package availability

rai_openset and rai_nomad are not yet available through pip. If your workflow relies on openset detection or NoMaD integration, please refer to the developer environment instructions setup.

rai_interfaces is available as apt package. However, due to package distribution delays, the latest version may not be immediately available. If you encounter missing imports, please build rai_interfaces from source.

RAI modules

RAI is a modular framework. You can install only the modules you need.

Module	Description	Documentation
rai-core	Core functionality	link
rai-whoami	Embodiment module	link
rai-s2s	Speech-to-Speech module	link
rai-sim	Simulation module	link
rai-bench	Benchmarking module	link

RAI outside of ROS 2

RAI can be used outside of ROS 2. This means that no ROS 2 related features will be available.

You can still use RAI's core agent framework, tool system, message passing, and integrations such as LangChain, even if ROS 2 is not installed or sourced on your machine. This is useful for:

Developing and testing AI logic, tools, and workflows independently of any robotics middleware
Running RAI agents in simulation or cloud environments where ROS 2 is not present
Using RAI as a generic multimodal agent framework for non-robotic applications

If you later decide to integrate with ROS 2, you can simply install and source ROS 2, and all ROS 2-specific RAI features (such as connectors, aggregators, and tools) will become available automatically.

Setting up developer environment¶

1.1 Install poetry¶

RAI uses Poetry for python packaging and dependency management. Install poetry with the following line:

curl -sSL https://install.python-poetry.org | python3 -

Alternatively, you can opt to do so by following the official docs.

1.2 Clone the repository:¶

git clone https://github.com/RobotecAI/rai.git
cd rai

1.3 Download rai_interfaces ¶

vcs import < ros_deps.repos

1.4 Create poetry virtual environment and install dependencies:¶

poetry install
rosdep install --from-paths src --ignore-src -r -y

Additional dependencies

RAI is modular. If you want to use features such as speech-to-speech, simulation and benchmarking suite, openset detection, or NoMaD integration, install additional dependencies:

poetry install --with openset,nomad,s2s,simbench # or `--all-groups` for full setup

Group Name	Description	Dependencies
s2s	Speech-to-Speech functionality	rai_asr, rai_tts
simbench	Simulation and benchmarking tools	rai_sim, rai_bench
openset	Open-set detection capabilities	groundingdino, groundedsam
nomad	Visual Navigation - NoMaD integration	visualnav_transformer
docs	Documentation-related dependencies	mkdocs, mkdocs-material, pymdown-extensions

1.5 Configure RAI¶

Run the configuration tool to set up your LLM vendor and other settings:

poetry run streamlit run src/rai_core/rai/frontend/configurator.py

Web browser

If the web browser does not open automatically, open the URL displayed in the terminal manually.

2. Build the project:¶

2.1 Build RAI workspace¶

colcon build --symlink-install

2.2 Activate the virtual environment:¶

source ./setup_shell.sh

3. Setting up vendors¶

RAI is vendor-agnostic. Use the configuration in config.toml to set up your vendor of choice for RAI modules. Vendor choices for RAI and our recommendations are summarized in Vendors Overview.

Best-performing AI models

We strongly recommend you to use of best-performing AI models to get the most out of RAI!

Pick your local solution or service provider and follow one of these guides:

Ollama
OpenAI
AWS Bedrock