Concourse is a simple and scalable CI system.
$ helm repo add concourse https://concourse-charts.storage.googleapis.com/
$ helm install my-release concourse/concourseThis chart bootstraps a Concourse deployment on a Kubernetes cluster using the Helm package manager.
- Kubernetes 1.6 (for
pod affinitysupport) PersistentVolumesupport on underlying infrastructure (if persistence is required)- Helm v3.x
To install the chart with the release name my-release:
$ helm install my-release concourse/concourseTo uninstall/delete the my-release deployment:
$ helm delete my-releaseThe command removes nearly all the Kubernetes components associated with the chart and deletes the release.
PS: By default, a namespace is created for the
mainteam named after${RELEASE}-mainand is kept untouched after ahelm delete. See the Configuration section for how to control the behavior.
This chart uses StatefulSets for Concourse Workers. Deleting a StatefulSet does not delete associated PersistentVolumes.
Do the following after deleting the chart release to clean up orphaned Persistent Volumes.
$ kubectl delete pvc -l app=${RELEASE-NAME}-workerIf a Worker isn't taking on work, you can recreate it with kubectl delete pod. This initiates a graceful shutdown by "retiring" the worker, to ensure Concourse doesn't try looking for old volumes on the new worker.
The valueworker.terminationGracePeriodSeconds can be used to provide an upper limit on graceful shutdown time before forcefully terminating the container.
Check the output of fly workers, and if a worker is stalled, you'll also need to run fly prune-worker to allow the new incarnation of the worker to start.
TIP: you can download
flyeither from https://concourse-ci.org/download.html or the home page of your Concourse installation.
When using ephemeral workers with worker.kind: Deployment and spawning a lot of (new) workers, you might run into issue 3091.
As a workaround you could start a worker.extraInitContainers to cleanup unused loopback devices.
By default, the worker's LivenessProbe will trigger a restart of the worker container if it detects errors when trying to reach the worker's healthcheck endpoint which takes care of making sure that the workers' components can properly serve their purpose.
See Configuration and values.yaml for the configuration of both the livenessProbe (worker.livenessProbe) and the default healthchecking timeout (concourse.worker.healthcheckTimeout).
The following table lists the configurable parameters of the Concourse chart and their default values.
For configurable Concourse parameters, refer to values.yaml' concourse section. All parameters under this section are strictly mapped from the concourse binary commands.
For example if one needs to configure the Concourse external URL, the param concourse -> web -> externalUrl should be set, which is equivalent to running the concourse binary as concourse web --external-url.
For those sub-sections that have enabled, one needs to set enabled to be true to use the following params within the section.
Specify each parameter using the --set key=value[,key=value] argument to helm install.
Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example,
$ helm install my-release -f values.yaml concourse/concourseTip: You can use the default values.yaml
For your convenience, this chart provides some default values for secrets, but it is recommended that you generate and manage these secrets outside the Helm chart.
To do that, set secrets.create to false, create files for each secret value, and turn it all into a Kubernetes Secret.
Be careful with introducing trailing newline characters; following the steps below ensures none end up in your secrets. First, perform the following to create the mandatory secret values:
# Create a directory to host the set of secrets that are
# required for a working Concourse installation and get
# into it.
#
mkdir concourse-secrets
cd concourse-secretsConcourse needs three sets of key-pairs in order to work:
- web key pair,
- worker key pair, and
- the session signing token.
You can generate all three key-pairs by following either of these two methods:
docker run -v $PWD:/keys --rm -it concourse/concourse generate-key -t rsa -f /keys/session-signing-key
docker run -v $PWD:/keys --rm -it concourse/concourse generate-key -t ssh -f /keys/worker-key
docker run -v $PWD:/keys --rm -it concourse/concourse generate-key -t ssh -f /keys/host-key
rm session-signing-key.pubssh-keygen -t rsa -f host-key -N '' -m PEM
ssh-keygen -t rsa -f worker-key -N '' -m PEM
ssh-keygen -t rsa -f session-signing-key -N '' -m PEM
rm session-signing-key.pubYou'll also need to create/copy secret values for optional features. See templates/web-secrets.yaml and templates/worker-secrets.yaml for possible values.
In the example below, we are not using the PostgreSQL chart dependency, and so we must set postgresql-user and postgresql-password secrets.
# Still within the directory where our secrets exist,
# copy a postgres user to clipboard and paste it to file.
#
printf "%s" "$(pbpaste)" > postgresql-user
# Copy a postgres password to clipboard and paste it to file
#
printf "%s" "$(pbpaste)" > postgresql-password
# Copy Github client id and secrets to clipboard and paste to files
#
printf "%s" "$(pbpaste)" > github-client-id
printf "%s" "$(pbpaste)" > github-client-secret
# Set an encryption key for DB encryption at rest
#
printf "%s" "$(openssl rand -base64 24)" > encryption-key
# Create a local user for concourse.
#
printf "%s:%s" "concourse" "$(openssl rand -base64 24)" > local-usersMake a directory for each secret and then move generated credentials into appropriate directories.
mkdir concourse web worker
# worker secrets
mv host-key.pub worker/host-key-pub
mv worker-key.pub worker/worker-key-pub
mv worker-key worker/worker-key
# web secrets
mv session-signing-key web/session-signing-key
mv host-key web/host-key
cp worker/worker-key-pub web/worker-key-pub
mv local-users web/local-users
# other concourse secrets (there may be more than the 3 listed below)
mv encryption-key concourse/encryption-key
mv postgresql-password concourse/postgresql-password
mv postgresql-user concourse/postgresql-userThen create the secrets from each of the 3 directories:
kubectl create secret generic [my-release]-worker --from-file=worker/
kubectl create secret generic [my-release]-web --from-file=web/
kubectl create secret generic [my-release]-concourse --from-file=concourse/Make sure you clean up after yourself.
This chart mounts a Persistent Volume for each Concourse Worker.
The volume is created using dynamic volume provisioning.
If you want to disable it or change the persistence properties, update the persistence section of your custom values.yaml file:
## Persistent Volume Storage configuration.
## ref: https://kubernetes.io/docs/user-guide/persistent-volumes
##
persistence:
## Enable persistence using Persistent Volume Claims.
##
enabled: true
## Worker Persistence configuration.
##
worker:
## Persistent Volume Storage Class.
##
storageClass: generic
## Persistent Volume Access Mode.
##
accessMode: ReadWriteOnce
## Persistent Volume Storage Size.
##
size: "20Gi"It is highly recommended to use Persistent Volumes for Concourse Workers; otherwise, the Concourse volumes managed by the Worker are stored in an emptyDir volume on the Kubernetes node's disk. This will interfere with Kubernete's ImageGC and the node's disk will fill up as a result.
If your cluster allows automatic creation/retrieval of TLS certificates (e.g. cert-manager), please refer to the documentation for that mechanism.
To manually configure TLS, first create/retrieve a key & certificate pair for the address(es) you wish to protect. Then create a TLS secret in the namespace:
kubectl create secret tls concourse-web-tls --cert=path/to/tls.cert --key=path/to/tls.keyInclude the secret's name, along with the desired hostnames, in the web.ingress.tls section of your custom values.yaml file:
## Configuration values for Concourse Web components.
##
web:
## Ingress configuration.
## ref: https://kubernetes.io/docs/user-guide/ingress/
##
ingress:
## Enable ingress.
##
enabled: true
## Hostnames.
## Either `hosts` or `rulesOverride` must be provided if Ingress is enabled.
## `hosts` sets up the Ingress with default rules per provided hostname.
##
hosts:
- concourse.domain.com
## Ingress rules override
## Either `hosts` or `rulesOverride` must be provided if Ingress is enabled.
## `rulesOverride` allows the user to define the full set of ingress rules, for more complex Ingress setups.
##
##
rulesOverride:
- host: concourse.domain.com
http:
paths:
- path: '/*'
backend:
serviceName: "ssl-redirect"
servicePort: "use-annotation"
- path: '/*'
backend:
serviceName: "concourse-web"
servicePort: atc
## TLS configuration.
## Secrets must be manually created in the namespace.
##
tls:
- secretName: concourse-web-tls
hosts:
- concourse.domain.comBy default, this chart deploys a single postgresql instance as a statefulSet, the conection details will be shared with concourse. You can change the connection details using the attributes of the postgresql.auth.
🚩The Postgres in this chart is not considered production-ready. It's good for getting started with deploying Concourse with Helm, but you should move to a managed PostgreSQL if your instance begins to scale up.
You can also bring your own PostgreSQL. To do so, set postgresql.enabled to false, and then configure Concourse's postgres values (concourse.web.postgres.*) See values.yaml.
Note that Concourse (by default) will attempt to read in some values directly from secrets. Therefore, these values do not have to be explicitly passed into Concourse as individual arguments, or as members of lists or files. (e.g. Part of values.yaml, etc.) Some examples of these secret values are: postgresql-user, postgresql-password (in secret "[my-release]-concourse" or "[my-release]-web"), and others. See templates/web-secrets.yaml for possible values and the secrets section on this README for guidance on how to set those secrets.
Pipelines usually need credentials to do things. Concourse supports the use of a Credential Manager so your pipelines can contain references to secrets instead of the actual secret values. You can't use more than one credential manager at a time.
By default, this chart uses Kubernetes Secrets as a credential manager.
For a given Concourse team, a pipeline looks for secrets in a namespace named [namespacePrefix][teamName]. The namespace prefix is the release name followed by a hyphen by default, and can be overridden with the value concourse.web.kubernetes.namespacePrefix. Each team listed under concourse.web.kubernetes.teams will have a namespace created for it, and the namespace remains after deletion of the release unless you set concourse.web.kubernetes.keepNamespace to false. By default, a namespace will be created for the main team.
The service account used by Concourse must have get access to secrets in that namespace. When rbac.create is true, this access is granted for each team listed under concourse.web.kubernetes.teams.
Here are some examples of the lookup heuristics, given release name concourse:
In team accounting-dev, pipeline my-app; the expression ((api-key)) resolves to:
- the secret value in namespace:
concourse-accounting-devsecret:my-app.api-key, key:value - and if not found, is the value in namespace:
concourse-accounting-devsecret:api-key, key:value
In team accounting-dev, pipeline my-app, the expression ((common-secrets.api-key)) resolves to:
- the secret value in namespace:
concourse-accounting-devsecret:my-app.common-secrets, key:api-key - and if not found, is the value in namespace:
concourse-accounting-devsecret:common-secrets, key:api-key
Be mindful of your team and pipeline names, to ensure they can be used in namespace and secret names, e.g. no underscores.
To test, create a secret in namespace concourse-main:
kubectl create secret generic hello --from-literal 'value=Hello world!'Then fly set-pipeline with the following pipeline, and trigger it:
jobs:
- name: hello-world
plan:
- task: say-hello
config:
platform: linux
image_resource:
type: docker-image
source: {repository: alpine}
params:
HELLO: ((hello))
run:
path: /bin/sh
args: ["-c", "echo $HELLO"]To use Vault, set concourse.web.kubernetes.enabled to false, and set the following values:
## Configuration values for the Credential Manager.
## ref: https://concourse-ci.org/creds.html
##
concourse:
web:
vault:
## Use Hashicorp Vault for the Credential Manager.
##
enabled: true
## URL pointing to vault addr (i.e. http://vault:8200).
##
url:
## vault path under which to namespace credential lookup, defaults to /concourse.
##
pathPrefix:To use Credhub, set concourse.web.kubernetes.enabled to false, and consider the following values:
## Configuration for using Credhub as a credential manager.
## Ref: https://concourse-ci.org/credhub-credential-manager.html
##
concourse:
web:
credhub:
## Enable the use of Credhub as a credential manager.
##
enabled: true
## CredHub server address used to access secrets
## Example: https://credhub.example.com
##
url:
## Path under which to namespace credential lookup. (default: /concourse)
##
pathPrefix:
## Enables using a CA Certificate
##
useCaCert: false
## Enables insecure SSL verification.
##
insecureSkipVerify: falseTo use Conjur, set concourse.web.kubernetes.enabled to false, and set the following values:
## Configuration for using Conjur as a credential manager.
## Ref: https://concourse-ci.org/conjur-credential-manager.html
##
concourse:
web:
conjur:
## Enable the use of Conjur as a credential manager.
##
enabled: true
## Conjur server address used to access secrets
## Example: https://conjur.example.com
##
applianceUrl:
## Base path used to locate a vault or safe-level secret
## Default: vaultName/{{.Secret}})
##
secretTemplate:
## Base path used to locate a team-level secret
## Default: concourse/{{.Team}}/{{.Secret}}
##
teamSecretTemplate:
## Base path used to locate a pipeline-level secret
## Default: concourse/{{.Team}}/{{.Pipeline}}/{{.Secret}}
##
pipelineSecretTemplate:
secrets:
# Org account.
conjurAccount:
# Host username. E.g host/concourse
conjurAuthnLogin:
# Api key related to the host.
conjurAuthnApiKey:
# Token file used if conjur instance is running in k8s or iam. E.g. /path/to/token_file
conjurAuthnTokenFile:
# CA Certificate to specify if conjur instance is deployed with a self-signed cert
conjurCACert:You can specify either conjurAuthnApiKey that corresponds to the Conjur host OR conjurAuthnTokenFile if running in K8s or IAM.
If your Conjur instance is deployed with a self-signed SSL certificate, you will need to set conjurCACert property in your values.yaml.
To use SSM, set concourse.web.kubernetes.enabled to false, and set concourse.web.awsSsm.enabled to true.
Authentication can be configured to use an access key and secret key as well as a session token. This is done by setting concourse.web.awsSsm.keyAuth.enabled to true. Alternatively, if it set to false, AWS IAM role based authentication (instance or pod credentials) is assumed. To use a session token, concourse.web.awsSsm.useSessionToken should be set to true. The secret values can be managed using the values specified in this helm chart or separately. For more details, see https://concourse-ci.org/creds.html#ssm.
For a given Concourse team, a pipeline looks for secrets in SSM using either /concourse/{team}/{secret} or /concourse/{team}/{pipeline}/{secret}; the patterns can be overridden using the concourse.web.awsSsm.teamSecretTemplate and concourse.web.awsSsm.pipelineSecretTemplate settings.
Concourse requires AWS credentials which are able to read from SSM for this feature to function. Credentials can be set in the secrets.awsSsm* settings; if your cluster is running in a different AWS region, you may also need to set concourse.web.awsSsm.region.
The minimum IAM policy you need to use SSM with Concourse is:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "kms:Decrypt",
"Resource": "<kms-key-arn>",
"Effect": "Allow"
},
{
"Action": "ssm:GetParameter*",
"Resource": "<...arn...>:parameter/concourse/*",
"Effect": "Allow"
}
]
}Where <kms-key-arn> is the ARN of the KMS key used to encrypt the secrets in Parameter Store, and the <...arn...> should be replaced with a correct ARN for your account and region's Parameter Store.
To use Secrets Manager, set concourse.web.kubernetes.enabled to false, and set concourse.web.awsSecretsManager.enabled to true.
Authentication can be configured to use an access key and secret key as well as a session token. This is done by setting concourse.web.awsSecretsManager.keyAuth.enabled to true. Alternatively, if it set to false, AWS IAM role based authentication (instance or pod credentials) is assumed. To use a session token, concourse.web.awsSecretsManager.useSessionToken should be set to true. The secret values can be managed using the values specified in this helm chart or separately. For more details, see https://concourse-ci.org/creds.html#asm.
For a given Concourse team, a pipeline looks for secrets in Secrets Manager using either /concourse/{team}/{secret} or /concourse/{team}/{pipeline}/{secret}; the patterns can be overridden using the concourse.web.awsSecretsManager.teamSecretTemplate and concourse.web.awsSecretsManager.pipelineSecretTemplate settings.
Concourse requires AWS credentials which are able to read from Secrets Manager for this feature to function. Credentials can be set in the secrets.awsSecretsmanager* settings; if your cluster is running in a different AWS region, you may also need to set concourse.web.awsSecretsManager.region.
The minimum IAM policy you need to use Secrets Manager with Concourse is:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAccessToSecretManagerParameters",
"Effect": "Allow",
"Action": [
"secretsmanager:ListSecrets"
],
"Resource": "*"
},
{
"Sid": "AllowAccessGetSecret",
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret"
],
"Resource": [
"arn:aws:secretsmanager:::secret:/concourse/*"
]
}
]
}When adding a new Concourse flag, don't assign a default value in the values.yml that mirrors a default set by the Concourse binary.
Instead, you may add a comment specifying the default, such as
## pipeline-specific template for SSM parameters, defaults to: /concourse/{{.Team}}/{{.Pipeline}}/{{.Secret}}
##
pipelineSecretTemplate:
This prevents the behaviour drifting from that of the binary in case the binary's default values change.
We understand that the comment stating the binary's default can become stale. The current solution is a suboptimal one. It may be improved in the future by generating a list of the default values from the binary.
When running unit tests for helm, from the root of the repository, you can simply run the following.
helm unittest -f test/unittest/**/*.yaml .If you are debugging specific tests, simply target the folder or yaml file you want to run tests on.
Folder
helm unittest -f test/unittest/gateway-apis/*.yaml .Specific Test Suite
helm unittest -f test/unittest/gateway-apis/web-route-test.yaml .