Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ This project uses Tinybird's [Service Data Sources](https://www.tinybird.co/docs
- Pipe endpoint usage: Analyze requests and identify unusual traffic patterns.
- Storage: Monitor storage usage to optimize resources and avoid limits.
- Jobs: Keep track of the status and performance of scheduled jobs.
- Cluster health: Export every metric tracked by `organization.metrics_logs` per cluster host.
- Data transfer: Track transferred bytes by provider, region, event, and transfer kind.
- Sinks: Monitor executions, errors, duration, CPU, rows, and bytes by Sink Pipe.
- LLM usage: Track requests, tokens, duration, and cost by feature and origin.
- Backups: Monitor backup counts and the status, duration, and size of the latest backup.
- Private Link: Track provider-reported usage and cost.

## Get started

Expand All @@ -37,6 +43,18 @@ tb --cloud deploy --template https://github.com/tinybirdco/tinybird-org-metrics-

Access the Prometheus metrics endpoint at `https://api.tinybird.co/v0/pipes/organization_metrics.prometheus`.

The combined endpoint includes all metrics. Each category is also available as an
individual Prometheus endpoint:

| Endpoint | Source |
| --- | --- |
| `cluster` | `organization.metrics_logs` |
| `data_transfer` | `organization.data_transfer` |
| `sinks` | `organization.sinks_ops_log` |
| `llm_usage` | `organization.llm_usage` |
| `backups` | `organization.backups` |
| `private_link` | `organization.private_link` |

- Replace `api.tinybird.co` with your Tinybird host if the workspace is in a different region. See [Regions and endpoints](https://www.tinybird.co/docs/api-reference#regions-and-endpoints).
- **Token**: for a quick check in development, you can use the pre-existing *admin `user@domain` Token* of an Organization admin to authenticate requests. But for production, create a dedicated token with minimal permissions using the [Tinybird CLI](https://www.tinybird.co/docs/forward/get-started/administration/tokens):

Expand Down
5 changes: 5 additions & 0 deletions TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
This Org Metrics Exporter template uses Tinybird's [Service Data Sources](https://www.tinybird.co/docs/monitoring/organizations#organization-service-data-sources) to aggregate and publish organizational metrics as endpoints in [Prometheus format](https://www.tinybird.co/docs/forward/work-with-data/publish-data/guides/consume-api-endpoints-in-prometheus-format) for quick integration with common monitoring tools.

It exports ingestion, request, storage, job, cluster, data transfer, Sink, LLM
usage, backup, and Private Link metrics. The `organization_metrics` endpoint
combines every category, while `cluster`, `data_transfer`, `sinks`,
`llm_usage`, `backups`, and `private_link` can be scraped independently.

## Set up the project

```bash
Expand Down
2 changes: 1 addition & 1 deletion datadog/tinybird_org_metrics.json

Large diffs are not rendered by default.

Loading
Loading