Helpful resources:

• https://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-APPLICATION
• https://logs.fr.cloud.gov
• https://dashboard.fr.cloud.gov

To login to cloud.gov, enter cf login -a api.fr.cloud.gov --sso

To set the organization and space to development, use cf target -o gsa-datagov -s development

Many commands in this document are specific to an application. If you run app_name=<your-app-name>, such as app_name=inventory (should be in either the manifest.yml file or vars.yml file), then you can run the commands as-is without replacing the ${app_name}

To test jq queries against a json services variable, you can get the json list of services by running cf env ${app_name}. Then, copy the relevant JSON and use this thread to paste the command into your local command line variable:

read -r -d '' VCAP_SERVICES << EOM
[[copy-pasted from previous command]]
EOM

Now, you can run test jq queries against a VCAP_SERVICES variable as the buildpack should have.

You can read more details in the cloud.gov docs.

• First make sure that you're able to ssh via cf ssh-enabled ${app_name}
• If it's enabled then run cf ssh ${app_name}
  • If the app uses the .profile, then use cf ssh ${app_name} -t -c "/tmp/lifecycle/launcher /home/vcap/app bash ''"
• Activate your application's environment by running /tmp/lifecycle/shell (see SSH Session Environment docs)
  • If it responds with Could not infer app directory, please provide one, should be able to either provide a path or navigate to the correct dir and run /tmp/lifecycle/shell .

You're ready to go!

cf CLI v7 includes the push --strategy rolling for rolling deployment. This performs a zero-downtime deployment for you following blue-green deployment.

Occasionally, the new version is unhealthy. While the old healthy application will continue to serve traffic you must manually rollback by cancelling the deployment.

$ cf cancel-deployment ${app_name}

The easiest way to deploy, and to validate your PR will deploy as expected, is to push your branch onto develop (remote). If you have a typical git setup, this can be accomplished by running git push origin feature/current-branch:develop. This will then commit your changes to the develop branch which is setup to auto deploy for most applications (see example here). You can then watch the deployment run via the github actions logs, or through your own command line via cf logs ${app_name}. This should run in parallel with tests, as there is no expectation that develop be in a working state. However, you should announce to the team on slack if you are pushing to develop on an app, so that the team doesn't step on each other.

We have 12 spaces:

• development
• development-egress
• development-ssb
• development-ssb-egress
• staging
• staging-egress
• management-staging
• management-staging-egress
• prod
• management-egress
• management
• prod-egress

Each space should have a ci-deployer service.

$ cf create-service cloud-gov-service-account space-deployer ci-deployer

Each repository should create their own deployer key.

$ cf create-service-key ci-deployer inventory-deployer

View the username and password to add as a CI secret for deploying.

$ cf service-key ci-deployer inventory-deployer

We typically store secrets necessary for an application in a User Provided Service. If necessary, they will typically be named ${app_name}-secrets, with a different user-provided-service for each application and each environment.

If you are creating a user-provided-service for the first time, follow the documentation to either provide a JSON file or simply provide the keys and input values via the command prompt. You'll need to bind the secrets to the service across each space; the simplest way is usually to add it to the services list in the manifest file (but can be done manually).

If you are updating a certificate, please follow the instructions here: https://github.com/GSA/data.gov/wiki/Add-Login.gov-as-Authentication#update-key

If you are updating a user-provided-service, we need to get the current configuration and change only the parts that we want/need. To find what the current user-provided-service looks like, run cf env ${app_name}. It should appear in the VCAP_SERVICES JSON object, something like ${app_name}-secrets. Copy the credentials JSON object part, and place it in a temporary local file (usually secrets.json is git ignored). Edit this file (add, update, remove, etc) until the object is correct. Then, use cf uups ${app_name}-secrets -p /path/to/file to update the configuration. Remember this will need to be done for each application and space separately (use cf target -s <development, staging, prod> to change space), and the temporary JSON file should be removed once the updates are done and confirmed.

There are some user-provided services that are used to share information between apps, such as sysadmin-users. This is used by both catalog and inventory to track the entire data.gov team emails and make each team member an admin. This list is final: it will remove/add CKAN admins according to the passed list, so removing a team member from the list removes their sysadmin priveleges (the next restart, which occurs at least once an hour).

There's 2 separate user-provided services bound to inventory sysadmin-users and inventory-secrets. When adding a new team member make sure to update sysadmin-users not inventory-secrets.

It's important to note that the configuration changes won't take effect until the desired app is restarted.

space_name=prod
app_name=catalog
cf allow-space-ssh ${space_name}
cf enable-ssh ${app_name}
cf rs ${app_name}

cf disable-ssh ${app_name}
cf disallow-space-ssh ${space_name}
cf rs ${app_name}

To get a feel for the various things that we use to get an app running on cloud.gov, we suggest you manually deploy either catalog.data.gov or inventory-app. This will require using a docker SOLR application instead of the provisioned service (when doing in a sandbox), as the provisioned service is custom created for data.gov. To deploy via docker as a separate app, see notes here and previous manifest file.

To upload to any s3 bucket provisioned by cloud.gov, you need to get the specific credentials in your shell. We've repurposed a script from datagov-ssb to do just that.

• Copy the generate-s3-creds script at datagov-helper-scripts to a local file
• Make it an executable shell script
• Run it in a shell with an active cloud.gov cf token, and pass the appropriate service-instance and service-key as arguments:
  • Ex. ./name-of-script.sh static-site-images datagov
• Copy the environment variables it outputs and paste them back into your shell.
• You should now have access to the s3 bucket and should be able to perform aws cli s3 operations:
  • Ex. for i in `ls ~/Desktop/s3-uploads`; do aws s3 cp $i s3://${BUCKET_NAME}; done;