Add Cloud Run worker sample (CloudRunId + OpenTelemetry plugins) (#376) · temporalio/samples-python@ae5df2f · GitHub
Skip to content

Commit ae5df2f

Browse files
authored
Add Cloud Run worker sample (CloudRunId + OpenTelemetry plugins) (#376)
* Add Cloud Run worker sample (CloudRunId + OpenTelemetry plugins) * Comment the collector transform processors * Note one-time setup steps in README * Fix collector component names (otlp exporter, resourcedetection) * Use canonical collector component names (otlp_grpc exporter, resource_detection) * Factor Cloud Run deploy steps into deploy.sh and add service account creation
1 parent 66cad6a commit ae5df2f

17 files changed

Lines changed: 1122 additions & 0 deletions

‎gcp/__init__.py‎

Whitespace-only changes.

‎gcp/cloud_run/.dockerignore‎

Lines changed: 7 additions & 0 deletions

‎gcp/cloud_run/Dockerfile‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# The git-pinned temporalio compiles the Rust core; drop this builder for a Python base once the plugin ships on PyPI.
4+
FROM rust:1.91.0-slim-bookworm AS builder
5+
6+
COPY --from=ghcr.io/astral-sh/uv:0.8.15 /uv /uvx /bin/
7+
8+
RUN apt-get update \
9+
&& apt-get install --no-install-recommends --yes \
10+
build-essential \
11+
ca-certificates \
12+
git \
13+
libprotobuf-dev \
14+
pkg-config \
15+
protobuf-compiler \
16+
python3 \
17+
python3-dev \
18+
&& rm -rf /var/lib/apt/lists/*
19+
20+
ENV UV_COMPILE_BYTECODE=1 \
21+
UV_LINK_MODE=copy \
22+
UV_PYTHON=/usr/bin/python3
23+
24+
WORKDIR /app
25+
COPY pyproject.toml ./
26+
RUN uv sync --no-dev
27+
28+
FROM debian:bookworm-slim
29+
30+
ENV PATH=/app/.venv/bin:$PATH \
31+
PYTHONDONTWRITEBYTECODE=1 \
32+
PYTHONUNBUFFERED=1
33+
34+
RUN apt-get update \
35+
&& apt-get install --no-install-recommends --yes ca-certificates python3 \
36+
&& rm -rf /var/lib/apt/lists/* \
37+
&& groupadd --system app \
38+
&& useradd --system --gid app --create-home app
39+
40+
WORKDIR /app
41+
COPY --from=builder /app/.venv /app/.venv
42+
# The build context is this sample directory (flat); recreate the package path.
43+
COPY --chown=app:app __init__.py settings.py worker.py workflow.py /app/gcp/cloud_run/
44+
RUN touch /app/gcp/__init__.py \
45+
&& chown -R app:app /app/gcp
46+
47+
USER app
48+
CMD ["python", "-m", "gcp.cloud_run.worker"]

‎gcp/cloud_run/README.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Google Cloud Run Worker
2+
3+
Run a Temporal Worker on a [Google Cloud Run worker
4+
pool](https://cloud.google.com/run/docs/worker-pools) with two GCP plugins on
5+
one client. `temporalio.contrib.gcp.cloud_run.opentelemetry.OpenTelemetryPlugin`
6+
exports Temporal Core metrics and traces over OTLP/gRPC to a Google-Built
7+
OpenTelemetry Collector sidecar, which forwards traces to the Cloud Telemetry API
8+
and metrics to Google Managed Service for Prometheus; endpoint, service name
9+
(from `CLOUD_RUN_WORKER_POOL`), tracer provider, and 60s metric export are plugin
10+
defaults, and `worker.py` opts into `add_temporal_spans=True`.
11+
`temporalio.contrib.gcp.cloud_run.id.CloudRunIdPlugin` sets the client identity
12+
to `<instance_id>@<revision>` from Cloud Run instance metadata, so each container
13+
is individually identifiable as a poller.
14+
15+
`CloudRunIdPlugin` is unreleased, so `pyproject.toml` pins `temporalio` to the
16+
SDK commit that adds it (a git source, which also builds inside the container);
17+
drop the `[tool.uv.sources]` override once it ships on PyPI.
18+
19+
Prerequisites: a Temporal Cloud namespace and API key; a Google Cloud project
20+
with billing and an authenticated `gcloud` CLI; `envsubst` (`gettext` package).
21+
Worker pools bill continuously, so scale to zero after testing (see below).
22+
23+
## Deploy
24+
25+
Run from the repository root. Set your own values (also used by the verify step
26+
below), then run the deploy script:
27+
28+
```bash
29+
export PROJECT_ID=your-project-id REGION=us-central1
30+
export REPOSITORY=temporal-workers WORKER_POOL=temporal-gcp-cloud-run
31+
export SERVICE_ACCOUNT_EMAIL="cloud-run-worker@${PROJECT_ID}.iam.gserviceaccount.com"
32+
export TEMPORAL_NAMESPACE=your-namespace.account-id
33+
export TEMPORAL_ADDRESS="${TEMPORAL_NAMESPACE}.tmprl.cloud:7233"
34+
export TEMPORAL_TASK_QUEUE=gcp-cloud-run
35+
export TEMPORAL_API_KEY_FILE=/secure/path/to/temporal-api-key
36+
export TEMPORAL_API_KEY_SECRET=temporal-api-key TEMPORAL_API_KEY_SECRET_VERSION=1
37+
export COLLECTOR_CONFIG_SECRET=temporal-otel-collector COLLECTOR_CONFIG_SECRET_VERSION=1
38+
export WORKER_IMAGE="${REGION}-docker.pkg.dev/${PROJECT_ID}/${REPOSITORY}/gcp-cloud-run:v1"
39+
export INSTANCE_COUNT=1
40+
41+
./gcp/cloud_run/deploy.sh
42+
```
43+
44+
`deploy.sh` enables the required APIs; creates the Artifact Registry repo, the
45+
runtime service account, and the API key and collector-config secrets; grants the
46+
telemetry and secret-access roles; then builds the image and deploys the
47+
two-container worker pool. The create steps are one-time and re-run safely.
48+
49+
Worker pools bill continuously, so scale to zero when done testing:
50+
51+
```bash
52+
gcloud run worker-pools update "$WORKER_POOL" --instances 0 \
53+
--region "$REGION" --project "$PROJECT_ID"
54+
```
55+
56+
## Run a Workflow and verify
57+
58+
```bash
59+
TEMPORAL_API_KEY="$(cat "$TEMPORAL_API_KEY_FILE")" \
60+
uv run python -m gcp.cloud_run.starter
61+
```
62+
63+
The starter prints `Hello, Temporal!`. In Google Cloud, Trace Explorer shows
64+
`RunWorkflow:GreetingWorkflow` (with `service.name` = the worker-pool name) and
65+
Metrics Explorer shows
66+
`prometheus.googleapis.com/temporal_workflow_completed_total/counter`. In the
67+
Temporal UI, the task queue's pollers report the `<instance_id>@<revision>`
68+
identity set by `CloudRunIdPlugin`.

‎gcp/cloud_run/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
"""Google Cloud Run worker-pool sample."""
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# @@@SNIPSTART python-cloud-run-collector-config
2+
receivers:
3+
otlp:
4+
protocols:
5+
grpc:
6+
endpoint: localhost:4317
7+
8+
processors:
9+
batch/traces:
10+
send_batch_max_size: 200
11+
send_batch_size: 200
12+
timeout: 5s
13+
memory_limiter:
14+
check_interval: 1s
15+
limit_percentage: 65
16+
spike_limit_percentage: 20
17+
resource_detection:
18+
detectors: [gcp]
19+
timeout: 10s
20+
# Rename Temporal datapoint labels that collide with the target labels Managed Service for Prometheus injects (e.g. namespace).
21+
transform/collision:
22+
metric_statements:
23+
- context: datapoint
24+
statements:
25+
- set(attributes["exported_location"], attributes["location"])
26+
- delete_key(attributes, "location")
27+
- set(attributes["exported_cluster"], attributes["cluster"])
28+
- delete_key(attributes, "cluster")
29+
- set(attributes["exported_namespace"], attributes["namespace"])
30+
- delete_key(attributes, "namespace")
31+
- set(attributes["exported_job"], attributes["job"])
32+
- delete_key(attributes, "job")
33+
- set(attributes["exported_instance"], attributes["instance"])
34+
- delete_key(attributes, "instance")
35+
- set(attributes["exported_project_id"], attributes["project_id"])
36+
- delete_key(attributes, "project_id")
37+
# The Telemetry API expects the Google Cloud project in gcp.project_id.
38+
transform/set_project_id:
39+
error_mode: ignore
40+
trace_statements:
41+
- set(resource.attributes["gcp.project_id"], resource.attributes["gcp.project.id"]) where resource.attributes["gcp.project.id"] != nil
42+
- set(resource.attributes["gcp.project_id"], resource.attributes["cloud.account.id"]) where resource.attributes["gcp.project_id"] == nil and resource.attributes["cloud.account.id"] != nil
43+
44+
exporters:
45+
googlemanagedprometheus:
46+
otlp_grpc:
47+
endpoint: telemetry.googleapis.com:443
48+
compression: none
49+
balancer_name: pick_first
50+
auth:
51+
authenticator: googleclientauth
52+
53+
extensions:
54+
googleclientauth:
55+
health_check:
56+
endpoint: 0.0.0.0:13133
57+
58+
service:
59+
extensions: [googleclientauth, health_check]
60+
pipelines:
61+
metrics:
62+
receivers: [otlp]
63+
processors: [memory_limiter, resource_detection, transform/collision]
64+
exporters: [googlemanagedprometheus]
65+
traces:
66+
receivers: [otlp]
67+
processors:
68+
[memory_limiter, resource_detection, transform/set_project_id, batch/traces]
69+
exporters: [otlp_grpc]
70+
# @@@SNIPEND

‎gcp/cloud_run/deploy.sh‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
#!/usr/bin/env bash
2+
# Deploy the Temporal Cloud Run worker sample. Export the variables documented in
3+
# README.md ("Deploy") first. The create steps are one-time and skipped when the
4+
# resource already exists, so this is safe to re-run to redeploy.
5+
set -euo pipefail
6+
7+
: "${PROJECT_ID:?set PROJECT_ID}"
8+
: "${REGION:?set REGION}"
9+
: "${REPOSITORY:?set REPOSITORY}"
10+
: "${SERVICE_ACCOUNT_EMAIL:?set SERVICE_ACCOUNT_EMAIL}"
11+
: "${TEMPORAL_API_KEY_SECRET:?set TEMPORAL_API_KEY_SECRET}"
12+
: "${TEMPORAL_API_KEY_FILE:?set TEMPORAL_API_KEY_FILE}"
13+
: "${COLLECTOR_CONFIG_SECRET:?set COLLECTOR_CONFIG_SECRET}"
14+
: "${WORKER_IMAGE:?set WORKER_IMAGE}"
15+
16+
# Enable APIs, create the Artifact Registry repo and runtime service account.
17+
gcloud services enable artifactregistry.googleapis.com cloudbuild.googleapis.com \
18+
monitoring.googleapis.com run.googleapis.com secretmanager.googleapis.com \
19+
telemetry.googleapis.com --project "$PROJECT_ID"
20+
gcloud artifacts repositories describe "$REPOSITORY" --location "$REGION" --project "$PROJECT_ID" >/dev/null 2>&1 ||
21+
gcloud artifacts repositories create "$REPOSITORY" --location "$REGION" \
22+
--repository-format docker --project "$PROJECT_ID"
23+
gcloud iam service-accounts describe "$SERVICE_ACCOUNT_EMAIL" --project "$PROJECT_ID" >/dev/null 2>&1 ||
24+
gcloud iam service-accounts create "${SERVICE_ACCOUNT_EMAIL%%@*}" --project "$PROJECT_ID"
25+
26+
# Grant the service account the collector's telemetry roles.
27+
for role in roles/logging.logWriter roles/monitoring.metricWriter roles/telemetry.tracesWriter; do
28+
gcloud projects add-iam-policy-binding "$PROJECT_ID" \
29+
--member "serviceAccount:${SERVICE_ACCOUNT_EMAIL}" --role "$role"
30+
done
31+
32+
# Store the API key and collector config as secrets the service account can read.
33+
gcloud secrets describe "$TEMPORAL_API_KEY_SECRET" --project "$PROJECT_ID" >/dev/null 2>&1 ||
34+
gcloud secrets create "$TEMPORAL_API_KEY_SECRET" --data-file "$TEMPORAL_API_KEY_FILE" --project "$PROJECT_ID"
35+
gcloud secrets describe "$COLLECTOR_CONFIG_SECRET" --project "$PROJECT_ID" >/dev/null 2>&1 ||
36+
gcloud secrets create "$COLLECTOR_CONFIG_SECRET" \
37+
--data-file gcp/cloud_run/collector-config.yaml --project "$PROJECT_ID"
38+
for secret in "$TEMPORAL_API_KEY_SECRET" "$COLLECTOR_CONFIG_SECRET"; do
39+
gcloud secrets add-iam-policy-binding "$secret" \
40+
--member "serviceAccount:${SERVICE_ACCOUNT_EMAIL}" \
41+
--role roles/secretmanager.secretAccessor --project "$PROJECT_ID"
42+
done
43+
44+
# Build the image (build context is the sample dir, keeping credentials out).
45+
gcloud builds submit gcp/cloud_run --region "$REGION" \
46+
--tag "$WORKER_IMAGE" --project "$PROJECT_ID"
47+
48+
# Render and deploy the two-container worker pool.
49+
envsubst < gcp/cloud_run/worker-pool.yaml > /tmp/worker-pool.yaml
50+
gcloud run worker-pools replace /tmp/worker-pool.yaml --project "$PROJECT_ID"

‎gcp/cloud_run/pyproject.toml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
[project]
2+
name = "temporalio-samples-gcp-cloud-run"
3+
version = "0.1a1"
4+
description = "Temporal Worker on a Google Cloud Run worker pool with the Cloud Run Id and OpenTelemetry plugins"
5+
authors = [{ name = "Temporal Technologies Inc", email = "sdk@temporal.io" }]
6+
requires-python = ">=3.10"
7+
readme = "README.md"
8+
license = "MIT"
9+
dependencies = ["temporalio[cloud-run-worker-otel]>=1.33.0,<2"]
10+
11+
[dependency-groups]
12+
dev = [
13+
"ruff>=0.5.0,<0.6",
14+
"mypy>=1.4.1,<2",
15+
]
16+
17+
[tool.uv]
18+
package = false
19+
20+
# The Cloud Run Id plugin is unreleased; pin temporalio to the SDK commit that adds it until it ships on PyPI.
21+
[tool.uv.sources]
22+
temporalio = { git = "https://github.com/temporalio/sdk-python", rev = "fa5cf46ea04920be138bafca7e20ca6208822c30" }

‎gcp/cloud_run/settings.py‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
"""Temporal connection settings shared by the worker and the starter.
2+
3+
Set ``TEMPORAL_API_KEY`` to connect to Temporal Cloud (which enables TLS); leave
4+
it unset for a plaintext dev server.
5+
"""
6+
7+
from __future__ import annotations
8+
9+
import os
10+
from dataclasses import dataclass
11+
12+
13+
@dataclass(frozen=True)
14+
class Settings:
15+
address: str
16+
namespace: str
17+
task_queue: str
18+
api_key: str | None
19+
20+
@property
21+
def tls(self) -> bool:
22+
return self.api_key is not None
23+
24+
25+
def load_settings() -> Settings:
26+
namespace = os.environ.get("TEMPORAL_NAMESPACE") or "default"
27+
api_key = os.environ.get("TEMPORAL_API_KEY")
28+
return Settings(
29+
address=os.environ.get("TEMPORAL_ADDRESS") or f"{namespace}.tmprl.cloud:7233",
30+
namespace=namespace,
31+
task_queue=os.environ.get("TEMPORAL_TASK_QUEUE") or "gcp-cloud-run",
32+
# Secret managers frequently preserve a trailing newline; strip it.
33+
api_key=api_key.strip() if api_key else None,
34+
)

‎gcp/cloud_run/starter.py‎

Lines changed: 34 additions & 0 deletions

0 commit comments

Comments
 (0)