Skip to content
chasty2Public

About

Self-hosted Pulumi -> Ansible -> Podman for web and data infrastructure. Feel free to use anything here that you find helpful

Resources

Stars

2 stars

Watchers

1 watching

Forks

Repository files navigation

CrowsNet

A self-hosted homelab that runs entirely out of infrastructure-as-code. Pulumi provisions virtual machines on Proxmox, Ansible configures the machines, and both run from a single container driven by the crowsnet.py CLI. Applications are run via rootless Podman containers. NFS storage of application data allows for VM's and containers to be destroyed and rebuilt with no data loss

Feel free to use anything here that you find helpful

Infrastructure

Three physical machines host everything. Proxmox VE runs the VMs, which are backed up to a separate machine running Proxmox Backup Server. A GPU workstation handles development and heavy compute

Host Type Description
esper Physical Proxmox VE hypervisor; NFS server for container data; nightly backups to PBS
simic Physical Proxmox Backup Server
abzan Physical GPU/development server
gate VM Public-facing SSH bastion
proxy VM Nginx Proxy Manager; terminates HTTP/HTTPS for public services
kube-1 VM MicroK8s node; runs FoundryVTT and Jellyfin
lab VM Development sandbox for concurrent Claude code instances
stage VM Ephemeral VM used only by the integration test suite

Getting Started

Requirements:

  • uv for Python tooling
  • podman to build and run the ops container
  • API access to the Proxmox host
  • Pulumi access token
  • Ansible vault password
git config core.hooksPath .githooks   # once per clone; blocks pushes to main
uv sync                               # host-side venv for the CLI and unit tests
./crowsnet.py build                   # build the ops container

Secrets are not committed in plaintext. Ansible reads its vault password from ansible/vault.pass and Pulumi reads its token from pulumi/pulumi.token; both are mounted into the container at run time

Provisioning (Pulumi)

VM definitions live in pulumi/apps/virtual_machines.py. VM's use MAC-associated fixed IP addresses that are assigned via DHCP. The stack name selects what gets deployed — k8s deploys workloads onto the MicroK8s cluster, every other stack deploys VM's:

Stack Purpose
stage Throwaway VM created and destroyed by the Molecule suite
dev Development sandbox
prod The live homelab
k8s MicroK8s workloads running on kube-1
./crowsnet.py deploy <stage|dev|prod|k8s>    # pulumi up
./crowsnet.py destroy <stage|dev|prod|k8s>   # pulumi destroy
./crowsnet.py refresh <stage|dev|prod|k8s>   # pulumi refresh

The k8s stack talks to the cluster through the kubernetes:kubeconfig stack config value rather than a kubeconfig file, so it needs to be set once:

cd pulumi
pulumi stack init k8s
pulumi config set --secret kubernetes:kubeconfig "$(ssh kube-1 microk8s config)" --stack k8s

Configuration (Ansible)

System-level configuration and deployment of app containers are managed via Ansible roles. Each role splits its work across up to five task files with matching tags — users, system, packages, services, and firewall — so any slice of the configuration can be applied on its own. The authoring conventions are in ansible/CLAUDE.md.

./crowsnet.py configure                    # full site deployment
./crowsnet.py configure --tags users       # run specific tags
./crowsnet.py configure --limit gate       # limit to specific host(s)
./crowsnet.py configure --check            # dry-run mode
./crowsnet.py update                       # patch and reboot all VMs

Ops Container

Ansible and Pulumi both run inside one Docker image. This allows for reproducible runs across different machines. This container is deployed from the CLI via crowsnet.py

Testing

The suite has two layers (a third, CI on a self-hosted runner, is still pending):

  • Unit (pytest) — fast, host-only, no infrastructure. Covers the Click CLI dispatch, podman command construction, and Pulumi component logic. Lives in tests/

  • Integration (Molecule) — provisions the real stage VM via Pulumi, converges a scenario, checks idempotency, runs tests, then tears the VM down. A scenario models a host and applies the same role stack that host gets in production; the assertions it runs live in each role's tests/ directory

./crowsnet.py test                                        # unit tests
./crowsnet.py test --integration                          # integration suite (scenario: common)
./crowsnet.py test --integration --scenario <scenario>    # integration suite for a specific scenario

CI/CD is done via GitHub Actions

About

Self-hosted Pulumi -> Ansible -> Podman for web and data infrastructure. Feel free to use anything here that you find helpful

Resources

Stars

2 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages