Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/docs/assets/logo-with-bg.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
466 changes: 247 additions & 219 deletions docs/docs/docker-compose-setup.md

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions docs/docs/exosphere/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

This document provides a comprehensive overview of Exosphere's top-level architecture, focusing on state execution, fanout mechanisms, and the unites keyword for stage unification.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Overview

Exosphere is built around a **state-based execution model** where workflows are composed of discrete states that can be executed independently. The architecture consists of several key components:
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/exosphere/create-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

The `Runtime` class is the core component that manages the execution environment for your Exosphere nodes. It handles node registration, state polling, execution, and communication with the state manager.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Runtime Setup

Before creating a runtime, you need to set up the state manager and configure your environment variables.
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/exosphere/dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

The Exosphere dashboard provides a comprehensive web interface for monitoring, debugging, and managing your workflows. This guide shows you how to set up and use the dashboard effectively.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Dashboard Overview

The Exosphere dashboard is a modern web application that connects to your state manager backend through secure server-side routes and provides:
Expand Down
90 changes: 90 additions & 0 deletions docs/docs/exosphere/local-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Local Setup Guide

This guide walks you through setting up Exosphere locally for development and testing. You'll learn how to run both the state manager and dashboard components, either using Docker Compose for quick setup or running them individually for more control.

## Overview

Exosphere consists of two main components that work together:

1. **State Manager** - Backend service that handles workflow execution and state management
2. **Dashboard** - Web interface for monitoring and managing workflows

You can set these up using:

- **Docker Compose** (recommended for quick start) - runs both services together
- **Individual Setup** - run each service separately for development

## Prerequisites

Before you begin, ensure you have:

- [Docker](https://docs.docker.com/get-docker/) installed (for Docker Compose approach)
- [Python 3.12+](https://www.python.org/downloads/) (for individual setup)
- [Node.js 18+](https://nodejs.org/) (for dashboard development)
- [MongoDB](https://www.mongodb.com/try/download/community) instance (cloud or local)

## Setup

=== "Docker Compose Setup (Recommended)"

The fastest way to get Exosphere running locally is using Docker Compose. This approach handles all the configuration and networking automatically.

Follow the guide: [Get Exosphere running locally with Docker](../docker-compose-setup.md)

=== "Individual Component Setup"

If you prefer more control over each component or want to develop locally, you can set up the state manager and dashboard separately.

To get Exosphere state manager running, follow [State Manager Setup](./state-manager-setup.md)

For complete monitoring dashboard setup details, see our [Dashboard Guide](./dashboard.md).


## Testing Your Setup

Once both services are running, test the complete setup:

1. **Verify state manager**:
```bash
curl http://localhost:8000/health
```

2. **Verify dashboard**:
```bash
curl http://localhost:3000
```

3. **Check API documentation**:
- Open http://localhost:8000/docs in your browser
- Test the `/health` endpoint
Comment on lines +58 to +59

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Avoid bare URL (MD034).

-- Open http://localhost:8000/docs in your browser
+- Open [http://localhost:8000/docs](http://localhost:8000/docs) in your browser
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- Open http://localhost:8000/docs in your browser
- Test the `/health` endpoint
- Open [http://localhost:8000/docs](http://localhost:8000/docs) in your browser
- Test the `/health` endpoint
🧰 Tools
🪛 LanguageTool

[grammar] ~58-~58: There might be a mistake here.
Context: ...tp://localhost:8000/docs in your browser - Test the /health endpoint ## Next Ste...

(QB_NEW_EN_OTHER)


[grammar] ~59-~59: There might be a mistake here.
Context: ...browser - Test the /health endpoint ## Next Steps With your local Exosphere in...

(QB_NEW_EN_OTHER)

🪛 markdownlint-cli2 (0.17.2)

58-58: Bare URL used

(MD034, no-bare-urls)

🤖 Prompt for AI Agents
In docs/docs/exosphere/local-setup.md around lines 58-59 the text contains a
bare URL ("http://localhost:8000/docs") which violates MD034; replace the bare
URL with a proper Markdown link or descriptive link text (for example: Open
[http://localhost:8000/docs](http://localhost:8000/docs) in your browser or Open
[Docs](/docs) in your browser) so the URL is not bare.


## Next Steps

With your local Exosphere instance running, you're ready to:

1. **[Register your first node](./register-node.md)** - Create custom processing logic
2. **[Create and run workflows](./create-graph.md)** - Build your first workflow
3. **[Monitor execution](./dashboard.md)** - Use the dashboard to track progress

## Troubleshooting

### Common Issues

**Port conflicts**: If ports 8000 or 3000 are already in use, modify the port mappings in your Docker commands or configuration files.

**MongoDB connection**: Ensure your MongoDB instance is accessible and the connection string is correct.

**Authentication errors**: Verify that `EXOSPHERE_API_KEY` matches `STATE_MANAGER_SECRET`.

**Dashboard can't connect**: Check that the state manager is running and accessible at the configured URI.

### Getting Help

- Check the [Docker Compose Setup Guide](../docker-compose-setup.md) for detailed configuration
- Review the [State Manager Setup Guide](./state-manager-setup.md) for backend troubleshooting
- Consult the [Dashboard Guide](./dashboard.md) for frontend issues
- Join our [Discord community](https://discord.com/invite/zT92CAgvkj) for support

## Production Considerations

For production deployment options, see our [hosted platform](https://exosphere.host) or contact our team for enterprise solutions.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Ensure trailing newline.

Add a single newline at EOF to satisfy MD047.

🧰 Tools
🪛 markdownlint-cli2 (0.17.2)

90-90: Files should end with a single newline character

(MD047, single-trailing-newline)

🤖 Prompt for AI Agents
In docs/docs/exosphere/local-setup.md around line 90 the file is missing a
trailing newline which violates MD047; open the file and add a single newline
character at the end of the file (ensure the final line ends with a newline)
then save so the file ends with exactly one trailing newline.

28 changes: 28 additions & 0 deletions docs/docs/exosphere/register-node.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Nodes are the building blocks of Exosphere workflows. Each node defines a specific piece of processing logic with typed inputs, outputs, and secrets. This guide shows you how to create and register custom nodes.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Node Structure

Every node inherits from `BaseNode` and defines three key components:
Expand All @@ -28,6 +30,32 @@ class MyNode(BaseNode):
pass
```


### Node Lifecycle Architecture

```mermaid
stateDiagram-v2
[*] --> Registered
Registered --> Ready
Ready --> Executing
Executing --> Completed
Executing --> Failed
Failed --> Ready
Completed --> Ready

state Executing {
[*] --> ValidateInputs
ValidateInputs --> ProcessData
ProcessData --> ValidateOutputs
ValidateOutputs --> [*]
}

note right of Failed
Automatic retry with
exponential backoff
end note
```
Comment on lines +34 to +57

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Surround heading and mermaid block with blank lines for markdownlint.

Avoid MD022/MD032 issues and ensure consistent rendering.

-### Node Lifecycle Architecture
+### Node Lifecycle Architecture
+
 ```mermaid
@@
     end note

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 LanguageTool</summary>

[grammar] ~34-~34: Use correct spacing
Context: ...ss ```   ### Node Lifecycle Architecture  ```mermaid stateDiagram-v2     [*] --> Registered     Registered --> Ready     Ready --> Executing     Executing --> Completed     Executing --> Failed     Failed --> Ready     Completed --> Ready          state Executing {         [*] --> ValidateInputs         ValidateInputs --> ProcessData         ProcessData --> ValidateOutputs         ValidateOutputs --> [*]     }          note right of Failed         Automatic retry with         exponential backoff     end note ```  ### Inputs  Define the data your node expect...

(QB_NEW_EN_OTHER_ERROR_IDS_5)

</details>
<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>

36-36: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below

(MD022, blanks-around-headings)

---

37-37: Lists should be surrounded by blank lines

(MD032, blanks-around-lists)

---

47-47: Inline HTML
Element: a

(MD033, no-inline-html)

---

48-48: Inline HTML
Element: img

(MD033, no-inline-html)

---

51-51: Multiple consecutive blank lines
Expected: 1; Actual: 2

(MD012, no-multiple-blanks)

</details>

</details>

<details>
<summary>🤖 Prompt for AI Agents</summary>

In docs/docs/exosphere/register-node.md around lines 34 to 57, the "Node
Lifecycle Architecture" heading and the mermaid fenced code block need
surrounding blank lines to satisfy markdownlint rules (MD022/MD032); add one
blank line before the "### Node Lifecycle Architecture" heading and one blank
line after the closing ``` of the mermaid block (remove any stray characters
like the shown "+" and ensure the file ends with a single newline) so the
heading and code block are isolated and render consistently.


</details>

<!-- fingerprinting:phantom:triton:chinchilla -->

<!-- This is an auto-generated comment by CodeRabbit -->


### Inputs

Define the data your node expects to receive:
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/exosphere/state-manager-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

This guide provides step-by-step instructions for setting up the Exosphere state manager in both local and production environments.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

## Overview

The Exosphere state manager is the core backend service that handles workflow execution, state management, and coordination between nodes. It provides a REST API for managing graph templates, registered nodes, and workflow execution states.
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/exosphere/trigger-graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Once you have created a graph template, you can trigger its execution by sending trigger states to the state manager. This guide shows you how to trigger graphs and monitor their execution.

> **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md).

The recommended way to trigger graphs is using the Exosphere Python SDK, which provides a clean interface to the State Manager API.

=== "Exosphere Python SDK"
Expand Down
51 changes: 34 additions & 17 deletions docs/docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,14 +159,6 @@ If you're working in a Jupyter notebook or Python REPL, consider these alternati
).start() # Blocks and runs forever
```

## Next Steps

Now that you have the basics, explore:

- **[Register Node](./exosphere/register-node.md)** - Understand how to create and register custom nodes
- **[Create Runtime](./exosphere/create-runtime.md)** - Learn how to set up and configure your runtime
- **[Create Graph](./exosphere/create-graph.md)** - Build workflows by connecting nodes together
- **[Trigger Graph](./exosphere/trigger-graph.md)** - Execute your workflows and monitor their progress

## Key Features

Expand All @@ -178,16 +170,41 @@ Now that you have the basics, explore:
- **Error Handling**: Built-in retry mechanisms and error recovery
- **Scalability**: Designed for high-volume batch processing and workflows

## Architecture
## Next Steps

```
+-------------------+ +--------------------+ +-------------------+
| Your Nodes | | Runtime | | State Manager |
| |<-->| |<-->| |
| • Inputs | | • Registration | | • Orchestration |
| • Outputs | | • Execution | | • State Mgmt |
| • Secrets | | • Error Handling | | • Dashboard |
+-------------------+ +--------------------+ +-------------------+
Now that you have the basics, explore:

- **[Local Setup](./exosphere/local-setup.md)** - Set up Exosphere locally for development and testing
- **[Register Node](./exosphere/register-node.md)** - Understand how to create and register custom nodes
- **[Create Runtime](./exosphere/create-runtime.md)** - Learn how to set up and configure your runtime
- **[Create Graph](./exosphere/create-graph.md)** - Build workflows by connecting nodes together
- **[Trigger Graph](./exosphere/trigger-graph.md)** - Execute your workflows and monitor their progress

Comment on lines +173 to +182

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Resolve extra blank lines to satisfy MD012 and improve flow.

-## Next Steps
-
-Now that you have the basics, explore:
+## Next Steps
+Now that you have the basics, explore:
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
## Next Steps
```
+-------------------+ +--------------------+ +-------------------+
| Your Nodes | | Runtime | | State Manager |
| |<-->| |<-->| |
| • Inputs | | • Registration | | • Orchestration |
| • Outputs | | • Execution | | • State Mgmt |
| • Secrets | | • Error Handling | | • Dashboard |
+-------------------+ +--------------------+ +-------------------+
Now that you have the basics, explore:
- **[Local Setup](./exosphere/local-setup.md)** - Set up Exosphere locally for development and testing
- **[Register Node](./exosphere/register-node.md)** - Understand how to create and register custom nodes
- **[Create Runtime](./exosphere/create-runtime.md)** - Learn how to set up and configure your runtime
- **[Create Graph](./exosphere/create-graph.md)** - Build workflows by connecting nodes together
- **[Trigger Graph](./exosphere/trigger-graph.md)** - Execute your workflows and monitor their progress
## Next Steps
Now that you have the basics, explore:
- **[Local Setup](./exosphere/local-setup.md)** - Set up Exosphere locally for development and testing
- **[Register Node](./exosphere/register-node.md)** - Understand how to create and register custom nodes
- **[Create Runtime](./exosphere/create-runtime.md)** - Learn how to set up and configure your runtime
- **[Create Graph](./exosphere/create-graph.md)** - Build workflows by connecting nodes together
- **[Trigger Graph](./exosphere/trigger-graph.md)** - Execute your workflows and monitor their progress
🧰 Tools
🪛 LanguageTool

[grammar] ~173-~173: Use correct spacing
Context: ... processing and workflows ## Next Steps Now that you have the basics, explore: ...

(QB_NEW_EN_OTHER_ERROR_IDS_5)


[grammar] ~175-~175: Use correct spacing
Context: ... Now that you have the basics, explore: - **Local Setup...

(QB_NEW_EN_OTHER_ERROR_IDS_5)


[grammar] ~177-~177: There might be a mistake here.
Context: ...here locally for development and testing - **[Register Node](./exosphere/register-node...

(QB_NEW_EN)


[grammar] ~178-~178: There might be a mistake here.
Context: ... how to create and register custom nodes - **[Create Runtime](./exosphere/create-runti...

(QB_NEW_EN)


[grammar] ~179-~179: There might be a mistake here.
Context: ...how to set up and configure your runtime - **[Create Graph](./exosphere/create-graph.m...

(QB_NEW_EN)


[grammar] ~180-~180: There might be a mistake here.
Context: ...d workflows by connecting nodes together - **[Trigger Graph](./exosphere/trigger-graph...

(QB_NEW_EN)


[grammar] ~181-~181: Use correct spacing
Context: ...our workflows and monitor their progress ### Data Flow Architecture ```mermaid seque...

(QB_NEW_EN_OTHER_ERROR_IDS_5)

🤖 Prompt for AI Agents
In docs/docs/getting-started.md around lines 173 to 182, there are extra blank
lines between the "Next Steps" heading and the subsequent list that trigger
MD012; remove the redundant blank lines so there's exactly one blank line
between the heading and the list and no consecutive blank lines inside the
block, preserving the existing list items and links and ensuring consistent
single spacing for proper Markdown linting and improved flow.


### Data Flow Architecture

```mermaid
sequenceDiagram
participant Client as Client Application
participant Runtime as Runtime
participant Node as Node Executor
participant StateMgr as State Manager
participant MongoDB as State Store

Client->>Runtime: Initialize with nodes
Runtime->>StateMgr: Register runtime & nodes
StateMgr->>MongoDB: Store registration info

loop Workflow Execution
StateMgr->>Runtime: Trigger node execution
Runtime->>Node: Execute node logic
Node->>Runtime: Return outputs
Runtime->>StateMgr: Update execution state
StateMgr->>MongoDB: Persist state changes
StateMgr->>Runtime: Trigger next node (if any)
end

StateMgr->>Client: Return final results
```

## Data Model (v1)
Expand Down
Loading