Quick setup guide¶
Before going further, make sure you have ROS 2 (jazzy or humble) installed and sourced on your system.
Docker images
RAI has experimental docker images. See the docker for instructions.
There are two ways to start using RAI:
-
Installing RAI using pip (recommended for end users)
-
Setting up a developer environment using poetry (recommended for developers)
Installing RAI¶
Virtual environment
We recommend installing RAI in a virtual environment (e.g., virtualenv, uv, or poetry) to keep your dependencies organized. Make sure to use the same version of python as the one used for ROS 2 (typically python3.10
for Humble and python3.12
for Jazzy).
If you plan to use ROS 2 commands (ros2 run
or ros2 launch
), you'll need to add your virtual environment's Python packages to your $PYTHONPATH
. This step is only necessary for ROS 2 integration - if you're just running RAI directly with Python, you can skip this step.
For reference, here's how to set this up when installing RAI from source: setup_shell.sh
-
Install core functionality:
pip install rai-core
-
Initialize the global configuration file:
rai-config-init
-
Optionally install ROS 2 dependencies:
sudo apt install ros-${ROS_DISTRO}-rai-interfaces
Package availability
rai_openset
and rai_nomad
are not yet available through pip. If your workflow relies on openset detection or NoMaD integration, please refer to the
developer environment instructions setup.
rai_interfaces
is available as apt
package. However, due to package distribution delays, the latest version may not be immediately available. If you encounter missing imports, please build rai_interfaces
from source.
RAI modules
RAI is a modular framework. You can install only the modules you need.
Module | Description | Documentation |
---|---|---|
rai-core | Core functionality | link |
rai-whoami | Embodiment module | link |
rai-s2s | Speech-to-Speech module | link |
rai-sim | Simulation module | link |
rai-bench | Benchmarking module | link |
RAI outside of ROS 2
RAI can be used outside of ROS 2. This means that no ROS 2 related features will be available.
You can still use RAI's core agent framework, tool system, message passing, and integrations such as LangChain, even if ROS 2 is not installed or sourced on your machine. This is useful for:
- Developing and testing AI logic, tools, and workflows independently of any robotics middleware
- Running RAI agents in simulation or cloud environments where ROS 2 is not present
- Using RAI as a generic multimodal agent framework for non-robotic applications
If you later decide to integrate with ROS 2, you can simply install and source ROS 2, and all ROS 2-specific RAI features (such as connectors, aggregators, and tools) will become available automatically.
Setting up developer environment¶
1.1 Install poetry¶
RAI uses Poetry for python packaging and dependency management. Install poetry with the following line:
curl -sSL https://install.python-poetry.org | python3 -
Alternatively, you can opt to do so by following the official docs.
1.2 Clone the repository:¶
git clone https://github.com/RobotecAI/rai.git
cd rai
1.3 Download rai_interfaces¶
vcs import < ros_deps.repos
1.4 Create poetry virtual environment and install dependencies:¶
poetry install
rosdep install --from-paths src --ignore-src -r -y
Additional dependencies
RAI is modular. If you want to use features such as speech-to-speech, simulation and benchmarking suite, openset detection, or NoMaD integration, install additional dependencies:
poetry install --with openset,nomad,s2s,simbench # or `--all-groups` for full setup
Group Name | Description | Dependencies |
---|---|---|
s2s | Speech-to-Speech functionality | rai_asr, rai_tts |
simbench | Simulation and benchmarking tools | rai_sim, rai_bench |
openset | Open-set detection capabilities | groundingdino, groundedsam |
nomad | Visual Navigation - NoMaD integration | visualnav_transformer |
docs | Documentation-related dependencies | mkdocs, mkdocs-material, pymdown-extensions |
1.5 Configure RAI¶
Run the configuration tool to set up your LLM vendor and other settings:
poetry run streamlit run src/rai_core/rai/frontend/configurator.py
Web browser
If the web browser does not open automatically, open the URL displayed in the terminal manually.
2. Build the project:¶
2.1 Build RAI workspace¶
colcon build --symlink-install
2.2 Activate the virtual environment:¶
source ./setup_shell.sh
3. Setting up vendors¶
RAI is vendor-agnostic. Use the configuration in config.toml
to set up your vendor
of choice for RAI modules. Vendor choices for RAI and our recommendations are summarized in
Vendors Overview.
Best-performing AI models
We strongly recommend you to use of best-performing AI models to get the most out of RAI!
Pick your local solution or service provider and follow one of these guides: