Skip to content

1_get-started.md

Latest commit

 

History

History
234 lines (162 loc) · 6.94 KB

1_get-started.md

File metadata and controls

234 lines (162 loc) · 6.94 KB
Raw
title Installation Guide
description Learn how to install, set up and configure OM1.
icon play

System Requirements

Operating System

  • Linux (Ubuntu 20, 22, 24)
  • MacOS 12.0+

Hardware

  • Sufficient memory to run vision and other models
  • Reliable WiFi or other networking
  • Sensors such as cameras, microphones, LIDAR units, IMUs
  • Actuators and outputs such as speakers, visual displays, and movement platforms (legs, arms, hands)
  • Hardware connected to the "central" computer via Zenoh, CycloneDDS, serial, usb, or custom APIs/libraries

Software

Ensure you have the following installed on your machine:

  • Go >= 1.25.0 (installation guide)
  • make build tool
  • portaudio for audio input and output
  • ffmpeg for video processing
  • Get your OpenMind API key here

Go Installation

# macOS
brew install go

# Linux - download from https://go.dev/dl/ or use your package manager
sudo apt-get update
sudo apt-get install golang-go

For other platforms, download from https://go.dev/dl/

PortAudio Library

For audio functionality, install portaudio:

# macOS
brew install portaudio

# Linux
sudo apt-get update
sudo apt-get install portaudio19-dev

ffmpeg

For video functionality, install FFmpeg:

# macOS
brew install ffmpeg

# Linux
sudo apt-get update
sudo apt-get install ffmpeg

To install Rust and Cargo (required for building SDKs like cdp-sdk), follow the steps below

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

CLI

OM1 provides a command-line interface (CLI). The main entry point is the om1 binary built from cmd/main.go which provides the following options:

CONFIG=[config_name] make run
  • config_name: Name of the config file (without .json5 extension) in the /config directory.

For development with debug logging:

CONFIG=[config_name] make dev

Installation and Setup

  1. Clone the repository

Run the following commands to clone the repository and set up the environment:

git clone https://github.com/OpenMind/OM1.git
cd OM1
make deps
make build

What these commands do:

  • make deps - Downloads and installs all Go module dependencies, fetches the zenoh-c library, and ensures your environment is ready for building.
  • make build - Compiles the OM1 binary from source code.

Dependencies are managed via Go modules (go.mod and go.sum).

Adding New Dependencies

To add a new Go package:

go get <package>    # Add the dependency
make deps           # Tidy and verify modules

Best Practices:

  • Keep dependencies minimal and prefer well-maintained packages
  • Run make check before committing (runs fmt, vet, lint, and test)
  • Use make fmt to format code and make lint to check for issues
  1. Set the configuration variables

Locate the config folder and add your OpenMind API key to /config/conversation.json5 (for example). If you do not already have one, you can obtain a free access key at https://portal.openmind.com/.

# /config/conversation.json5
...
"api_key": "om1_live_..."
...

Or, create a .env file in the project directory and add the following:

Note: Using the placeholder key openmind_free will generate errors.

OM_API_KEY=om1_live_...
  1. Run the Conversation Agent

Run the following command to start the Conversation Agent:

CONFIG=conversation make run

Note: Agent configuration names are only required when switching between different agents.

The conversation agent is just an example agent configuration.

If you want to interact with the agent and see how it works, make sure ASR and TTS are configured in conversation.json5.

ASR configuration (check in agent_inputs)

{
      "type": "GoogleASRInput"
}

TTS configuration (check in agent_actions)

{
      name: "speak",
      llm_label: "speak",
      connector: "elevenlabs_tts",
      config:
      {
        voice_id: "i4CzbCVWoqvD0P1QJCUL",
        "silence_rate": 20,
      },
}

During the first build, the system will automatically download the zenoh-c library and resolve all Go dependencies. This process may take several minutes to complete.

Prometheus and Grafana Monitoring

If you have Docker installed, you can use the included Docker Compose configuration to spin up Grafana and Prometheus to monitor real-time AI pipeline metrics (such as LLM response times and ASR latencies).

Run the following command:

docker-compose up -d grafana prometheus

Then navigate to http://localhost:3000 (default login: admin/admin). The OM1 Latency Monitoring dashboard is automatically provisioned and will display your latency metrics as you interact with the agent.

Understanding the Log Data

OM1 writes structured (JSON) logs using zap. At the default info level you see the high-level loop; run with make dev (or -log-level debug) for the full prompt, LLM response, and per-action detail.

The logs show one turn of the core loop — how the conversation agent senses its environment, calls the LLM, and dispatches actions:

  • Inputs (e.g. GoogleASRInput for speech, VLMGemini for vision) produce observations.
  • The State Fuser combines them into a single natural-language prompt.
  • The cortex ticks: it sends that prompt to the configured cortex_llm and receives a set of action calls.
  • Actions (e.g. speak, emotion) are dispatched to their connectors.
  • Latencies are exported to Prometheus/Grafana (see below).

A representative run looks like this (fields abbreviated):

{"level":"info","ts":1749590400.12,"caller":"runtime/runtime.go","msg":"cortex loop started"}
{"level":"info","ts":1749590401.34,"caller":"runtime/runtime.go","msg":"cortex tick","mode":"conversation","prompt":"You see a person in front of you. They said: \"Give me your paw!\""}
{"level":"debug","ts":1749590401.98,"caller":"llm/gemini.go","msg":"llm response","actions":[{"name":"speak","value":"Hello, let's shake paws!"},{"name":"emotion","value":"happy"}]}
{"level":"debug","ts":1749590402.01,"caller":"actions/action.go","msg":"dispatching action","name":"speak","connector":"elevenlabs_tts"}

Note: The exact prompt and actions depend on your agent config (system_prompt_base, agent_inputs, agent_actions) and on what the sensors observe. The conversation agent ships as a friendly dog named "Spot".

More Examples

There are more pre-configured agents in the /config folder. They can be run with the following command:

For example, to run the greeting_conversation agent:

CONFIG=greeting_conversation make run

If you configure a custom agent, replace <agent_name> with your agent and run the below command:

CONFIG=<agent_name> make run

To get started with development, refer here