-
Notifications
You must be signed in to change notification settings - Fork 45
Docs update for setup #347
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
40cb6ac
7e123c8
49e3d97
21120cd
aa0a8a7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| # Local Setup Guide | ||
|
|
||
| This guide walks you through setting up Exosphere locally for development and testing. You'll learn how to run both the state manager and dashboard components, either using Docker Compose for quick setup or running them individually for more control. | ||
|
|
||
| ## Overview | ||
|
|
||
| Exosphere consists of two main components that work together: | ||
|
|
||
| 1. **State Manager** - Backend service that handles workflow execution and state management | ||
| 2. **Dashboard** - Web interface for monitoring and managing workflows | ||
|
|
||
| You can set these up using: | ||
|
|
||
| - **Docker Compose** (recommended for quick start) - runs both services together | ||
| - **Individual Setup** - run each service separately for development | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before you begin, ensure you have: | ||
|
|
||
| - [Docker](https://docs.docker.com/get-docker/) installed (for Docker Compose approach) | ||
| - [Python 3.12+](https://www.python.org/downloads/) (for individual setup) | ||
| - [Node.js 18+](https://nodejs.org/) (for dashboard development) | ||
| - [MongoDB](https://www.mongodb.com/try/download/community) instance (cloud or local) | ||
|
|
||
| ## Setup | ||
|
|
||
| === "Docker Compose Setup (Recommended)" | ||
|
|
||
| The fastest way to get Exosphere running locally is using Docker Compose. This approach handles all the configuration and networking automatically. | ||
|
|
||
| Follow the guide: [Get Exosphere running locally with Docker](../docker-compose-setup.md) | ||
|
|
||
| === "Individual Component Setup" | ||
|
|
||
| If you prefer more control over each component or want to develop locally, you can set up the state manager and dashboard separately. | ||
|
|
||
| To get Exosphere state manager running, follow [State Manager Setup](./state-manager-setup.md) | ||
|
|
||
| For complete monitoring dashboard setup details, see our [Dashboard Guide](./dashboard.md). | ||
|
|
||
|
|
||
| ## Testing Your Setup | ||
|
|
||
| Once both services are running, test the complete setup: | ||
|
|
||
| 1. **Verify state manager**: | ||
| ```bash | ||
| curl http://localhost:8000/health | ||
| ``` | ||
|
|
||
| 2. **Verify dashboard**: | ||
| ```bash | ||
| curl http://localhost:3000 | ||
| ``` | ||
|
|
||
| 3. **Check API documentation**: | ||
| - Open http://localhost:8000/docs in your browser | ||
| - Test the `/health` endpoint | ||
|
|
||
| ## Next Steps | ||
|
|
||
| With your local Exosphere instance running, you're ready to: | ||
|
|
||
| 1. **[Register your first node](./register-node.md)** - Create custom processing logic | ||
| 2. **[Create and run workflows](./create-graph.md)** - Build your first workflow | ||
| 3. **[Monitor execution](./dashboard.md)** - Use the dashboard to track progress | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Common Issues | ||
|
|
||
| **Port conflicts**: If ports 8000 or 3000 are already in use, modify the port mappings in your Docker commands or configuration files. | ||
|
|
||
| **MongoDB connection**: Ensure your MongoDB instance is accessible and the connection string is correct. | ||
|
|
||
| **Authentication errors**: Verify that `EXOSPHERE_API_KEY` matches `STATE_MANAGER_SECRET`. | ||
|
|
||
| **Dashboard can't connect**: Check that the state manager is running and accessible at the configured URI. | ||
|
|
||
| ### Getting Help | ||
|
|
||
| - Check the [Docker Compose Setup Guide](../docker-compose-setup.md) for detailed configuration | ||
| - Review the [State Manager Setup Guide](./state-manager-setup.md) for backend troubleshooting | ||
| - Consult the [Dashboard Guide](./dashboard.md) for frontend issues | ||
| - Join our [Discord community](https://discord.com/invite/zT92CAgvkj) for support | ||
|
|
||
| ## Production Considerations | ||
|
|
||
| For production deployment options, see our [hosted platform](https://exosphere.host) or contact our team for enterprise solutions. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick (assertive) Ensure trailing newline. Add a single newline at EOF to satisfy MD047. 🧰 Tools🪛 markdownlint-cli2 (0.17.2)90-90: Files should end with a single newline character (MD047, single-trailing-newline) 🤖 Prompt for AI Agents |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,6 +2,8 @@ | |
|
|
||
| Nodes are the building blocks of Exosphere workflows. Each node defines a specific piece of processing logic with typed inputs, outputs, and secrets. This guide shows you how to create and register custom nodes. | ||
|
|
||
| > **📚 Getting Started**: For a complete local setup guide covering both the state manager and dashboard, see our [Local Setup Guide](./local-setup.md). | ||
|
|
||
| ## Node Structure | ||
|
|
||
| Every node inherits from `BaseNode` and defines three key components: | ||
|
|
@@ -28,6 +30,32 @@ class MyNode(BaseNode): | |
| pass | ||
| ``` | ||
|
|
||
|
|
||
| ### Node Lifecycle Architecture | ||
|
|
||
| ```mermaid | ||
| stateDiagram-v2 | ||
| [*] --> Registered | ||
| Registered --> Ready | ||
| Ready --> Executing | ||
| Executing --> Completed | ||
| Executing --> Failed | ||
| Failed --> Ready | ||
| Completed --> Ready | ||
|
|
||
| state Executing { | ||
| [*] --> ValidateInputs | ||
| ValidateInputs --> ProcessData | ||
| ProcessData --> ValidateOutputs | ||
| ValidateOutputs --> [*] | ||
| } | ||
|
|
||
| note right of Failed | ||
| Automatic retry with | ||
| exponential backoff | ||
| end note | ||
| ``` | ||
|
Comment on lines
+34
to
+57
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick (assertive) Surround heading and mermaid block with blank lines for markdownlint. Avoid MD022/MD032 issues and ensure consistent rendering. -### Node Lifecycle Architecture
+### Node Lifecycle Architecture
+
```mermaid
@@
end noteIn docs/docs/exosphere/register-node.md around lines 34 to 57, the "Node |
||
|
|
||
| ### Inputs | ||
|
|
||
| Define the data your node expects to receive: | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -159,14 +159,6 @@ If you're working in a Jupyter notebook or Python REPL, consider these alternati | |||||||||||||||||||||||||||||||||||||||||||||||||||
| ).start() # Blocks and runs forever | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| ``` | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| ## Next Steps | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| Now that you have the basics, explore: | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Register Node](./exosphere/register-node.md)** - Understand how to create and register custom nodes | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Create Runtime](./exosphere/create-runtime.md)** - Learn how to set up and configure your runtime | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Create Graph](./exosphere/create-graph.md)** - Build workflows by connecting nodes together | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Trigger Graph](./exosphere/trigger-graph.md)** - Execute your workflows and monitor their progress | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| ## Key Features | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
@@ -178,16 +170,41 @@ Now that you have the basics, explore: | |||||||||||||||||||||||||||||||||||||||||||||||||||
| - **Error Handling**: Built-in retry mechanisms and error recovery | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **Scalability**: Designed for high-volume batch processing and workflows | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| ## Architecture | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| ## Next Steps | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| ``` | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| +-------------------+ +--------------------+ +-------------------+ | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| | Your Nodes | | Runtime | | State Manager | | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| | |<-->| |<-->| | | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| | • Inputs | | • Registration | | • Orchestration | | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| | • Outputs | | • Execution | | • State Mgmt | | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| | • Secrets | | • Error Handling | | • Dashboard | | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| +-------------------+ +--------------------+ +-------------------+ | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Now that you have the basics, explore: | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Local Setup](./exosphere/local-setup.md)** - Set up Exosphere locally for development and testing | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Register Node](./exosphere/register-node.md)** - Understand how to create and register custom nodes | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Create Runtime](./exosphere/create-runtime.md)** - Learn how to set up and configure your runtime | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Create Graph](./exosphere/create-graph.md)** - Build workflows by connecting nodes together | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| - **[Trigger Graph](./exosphere/trigger-graph.md)** - Execute your workflows and monitor their progress | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Comment on lines
+173
to
+182
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick (assertive) Resolve extra blank lines to satisfy MD012 and improve flow. -## Next Steps
-
-Now that you have the basics, explore:
+## Next Steps
+Now that you have the basics, explore:📝 Committable suggestion
Suggested change
🧰 Tools🪛 LanguageTool[grammar] ~173-~173: Use correct spacing (QB_NEW_EN_OTHER_ERROR_IDS_5) [grammar] ~175-~175: Use correct spacing (QB_NEW_EN_OTHER_ERROR_IDS_5) [grammar] ~177-~177: There might be a mistake here. (QB_NEW_EN) [grammar] ~178-~178: There might be a mistake here. (QB_NEW_EN) [grammar] ~179-~179: There might be a mistake here. (QB_NEW_EN) [grammar] ~180-~180: There might be a mistake here. (QB_NEW_EN) [grammar] ~181-~181: Use correct spacing (QB_NEW_EN_OTHER_ERROR_IDS_5) 🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| ### Data Flow Architecture | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| ```mermaid | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| sequenceDiagram | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| participant Client as Client Application | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| participant Runtime as Runtime | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| participant Node as Node Executor | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| participant StateMgr as State Manager | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| participant MongoDB as State Store | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| Client->>Runtime: Initialize with nodes | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Runtime->>StateMgr: Register runtime & nodes | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| StateMgr->>MongoDB: Store registration info | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| loop Workflow Execution | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| StateMgr->>Runtime: Trigger node execution | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Runtime->>Node: Execute node logic | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Node->>Runtime: Return outputs | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| Runtime->>StateMgr: Update execution state | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| StateMgr->>MongoDB: Persist state changes | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| StateMgr->>Runtime: Trigger next node (if any) | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| end | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| StateMgr->>Client: Return final results | ||||||||||||||||||||||||||||||||||||||||||||||||||||
| ``` | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
| ## Data Model (v1) | ||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Nitpick (assertive)
Avoid bare URL (MD034).
📝 Committable suggestion
🧰 Tools
🪛 LanguageTool
[grammar] ~58-~58: There might be a mistake here.
Context: ...tp://localhost:8000/docs in your browser - Test the
/healthendpoint ## Next Ste...(QB_NEW_EN_OTHER)
[grammar] ~59-~59: There might be a mistake here.
Context: ...browser - Test the
/healthendpoint ## Next Steps With your local Exosphere in...(QB_NEW_EN_OTHER)
🪛 markdownlint-cli2 (0.17.2)
58-58: Bare URL used
(MD034, no-bare-urls)
🤖 Prompt for AI Agents