Send gtm-sdk telemetry to Grafana Cloud via otel-collector

Endpoints

Grafana Cloud exposes a region-scoped OTLP gateway:

https://otlp-gateway-prod-<REGION>.grafana.net/otlp

Find your exact URL on the Grafana Cloud “OpenTelemetry” (OTLP) config page for your stack — for example https://otlp-gateway-prod-us-east-3.grafana.net/otlp (the collector’s hard-coded default) or https://otlp-gateway-prod-eu-west-2.grafana.net/otlp. The collector strips any trailing /v1/traces / /v1/logs you paste, so either the base (.../otlp) or a full-signal URL works.

Authentication

Unlike Dash0 and Logfire (which use Bearer) or HyperDX (which expects the raw key with no prefix), Grafana Cloud’s OTLP gateway authenticates with HTTP Basic auth. Sending a Bearer header returns a 401.

Authorization: Basic base64("<INSTANCE_ID>:<TOKEN>")

<INSTANCE_ID> — the numeric OTLP-gateway instance / user id shown on the Grafana Cloud “OpenTelemetry” config page. This is not the org id embedded in the glc_ token.
<TOKEN> — a Grafana Cloud access-policy token (starts with glc_) scoped to write metrics, logs, and traces.

You do not encode this yourself. Set the two raw values as Infisical secrets and the collector derives the base64 credential (GRAFANA_OTLP_AUTH) at deploy time (see _grafana_basic_auth / _collector_secret_payload in src/otel_collector.py), so the raw glc_ token never reaches the collector container or the rendered config.

Setup

Add secrets to Infisical

Add these secrets to your Infisical project (prod environment):

Name	Value	Required
`GRAFANA_INSTANCE_ID`	The numeric OTLP-gateway instance id	Yes
`GRAFANA_API_KEY`	Your Grafana Cloud access-policy token (starts with `glc_`)	Yes
`GRAFANA_OTLP_ENDPOINT`	Your regional OTLP gateway URL	No — omit to use the hard-coded default (`us-east-3`)

Set them via the CLI:

# From the gtm-sdk repo root (its own .env.local holds INFISICAL_TOKEN + INFISICAL_PROJECT_ID)
set -a && source .env.local && set +a

infisical secrets set \
  GRAFANA_INSTANCE_ID "<YOUR_GRAFANA_INSTANCE_ID>" \
  --env=prod \
  --projectId "$INFISICAL_PROJECT_ID" \
  --token "$INFISICAL_TOKEN"

infisical secrets set \
  GRAFANA_API_KEY "<YOUR_GRAFANA_ACCESS_POLICY_TOKEN>" \
  --env=prod \
  --projectId "$INFISICAL_PROJECT_ID" \
  --token "$INFISICAL_TOKEN"

# Optional — only if you are not on the us-east-3 default region
infisical secrets set \
  GRAFANA_OTLP_ENDPOINT "https://otlp-gateway-prod-<REGION>.grafana.net/otlp" \
  --env=prod \
  --projectId "$INFISICAL_PROJECT_ID" \
  --token "$INFISICAL_TOKEN"

Deploy the collector

Once secrets are set, deploy the otel-collector:

# From the gtm-sdk repo root
set -a && source .env.local && set +a

infisical run --projectId "$INFISICAL_PROJECT_ID" --token "$INFISICAL_TOKEN" --env=prod \
    -- uv run modal deploy src/otel_collector.py

At deploy time the raw GRAFANA_INSTANCE_ID + GRAFANA_API_KEY are collapsed into the pre-encoded GRAFANA_OTLP_AUTH Basic credential and embedded in the collector’s secret, so when the container starts Grafana is already configured.

Verify the configuration

Check that Grafana was included in the collector config:

uv run python -c "
from src.otel_collector import build_collector_config
config = build_collector_config()
exporters = config.get('exporters', {})
print('Exporters:', list(exporters.keys()))
if 'otlphttp/grafana' in exporters:
    print('✓ Grafana is configured')
else:
    print('✗ Grafana is NOT configured')
"

How Grafana fits the collector

Grafana is one of the fan-out targets of the shared OTEL collector: apps spawn the collector Modal function over RPC, a localhost otelcol sidecar handles retry, queueing, and fan-out, and the Grafana exporter (otlphttp/grafana, Basic auth) ships alongside Dash0, HyperDX, and Logfire. See /telemetry/overview for the full architecture, and libs/telemetry.py / src/otel_collector.py for implementation details.

Troubleshooting

Secrets not found

Verify the secrets exist in Infisical:

# From the gtm-sdk repo root
set -a && source .env.local && set +a
infisical secrets get GRAFANA_API_KEY --env=prod \
  --projectId "$INFISICAL_PROJECT_ID" \
  --token "$INFISICAL_TOKEN"

401 / 403 from the gateway

Grafana Cloud uses Basic auth, not Bearer — a Bearer header returns 401. The collector handles this automatically; if you see 401s, re-check that GRAFANA_INSTANCE_ID is the numeric OTLP-gateway instance id (not the org id) and that the glc_ token’s access policy has write scope for the signals.
Confirm the endpoint region matches the stack the token belongs to.

Collector not starting

Check the Modal logs for the otel-collector app:

infisical run --projectId "$INFISICAL_PROJECT_ID" --token "$INFISICAL_TOKEN" --env=prod \
    -- uv run modal app logs otel-collector

No telemetry in Grafana

Verify the collector is running and Grafana is in the exporter list (see the verification step above).
Check Modal logs for export errors.
Verify the OTLP endpoint region is reachable and matches the token’s stack.

Mixing Grafana with other providers

You can enable multiple providers simultaneously — the collector fans out to all configured exporters. Grafana coexists with Dash0, HyperDX, and Logfire.

Reference

Grafana Cloud OTLP endpoint docs
OpenTelemetry Collector OTLPHTTP Exporter
src/otel_collector.py — collector app and config builder
libs/telemetry.py — app-side telemetry initialization

Get started

Concepts

Telemetry

Contributing

Changelog

Send gtm-sdk telemetry to Grafana Cloud via otel-collector

Endpoints

Authentication

Setup

How Grafana fits the collector

Troubleshooting

Reference

​Endpoints

​Authentication

​Setup

​How Grafana fits the collector

​Troubleshooting

​Reference

Endpoints

Authentication

Setup

How Grafana fits the collector

Troubleshooting

Reference