ROX-33124: Support roxie-based deployment with deploy script by mclasmeier · Pull Request #20731 · stackrox/stackrox · GitHub
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ROXIE_VERSION
184 changes: 180 additions & 4 deletions deploy/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,185 @@
# Deploy scripts
# Deploying StackRox Dev Versions

## Usage
## Deploying with roxie (recommended)

The legacy deploy scripts have served us well, but over time their configuration
grew into a maze of environment variables scattered across many files, each
subtly influencing deployment behavior. This makes them hard to maintain,
hard to debug, and hard to reproduce deployment configurations easily.

**[roxie](https://github.com/stackrox/roxie)** is the ACS/StackRox deployment
tool intended to replace the deployment shell scripts entirely.

Its core idea is simple: It uses **one self-contained configuration
file for reproducible deployments**, instead of implicitly picking up dozens of
environment variables.

### Running roxie directly

The recommended way is to install roxie natively on your machine (see
https://github.com/stackrox/roxie for installation instructions) and use it
directly from the command line, e.g.

```bash
roxie deploy --tag <stackrox main image tag>
```

for deploying the whole stack consisting of the operator, Central and SecuredCluster.

#### Shell script wrapper

For addressing muscle memory habits, `deploy.sh` can invoke roxie under the hood when you
opt-in into this behavior:

```bash
export USE_ROXIE_DEPLOY=true
./deploy/deploy.sh
```
Comment thread
coderabbitai[bot] marked this conversation as resolved.

Any extra arguments you pass to `deploy.sh` are forwarded directly to roxie,
so you can use roxie CLI flags even with `deploy.sh`, e.g.:

```bash
./deploy/deploy.sh --tag 4.11.0
```

The roxie backend in `deploy.sh` is **opt-in** for now. Over time, we expect it to
become opt-out, and eventually the legacy deployment scripts will be removed entirely.

### Configuration: environment variables vs. config file

The old scripts picked up configuration from the shell environment -- feature
flags, image tags, storage settings, and more. This is convenient at first, but
becomes a maintenance burden: it's unclear which variables are active, what
their defaults are, and whether your colleague's deployment matches yours.

With roxie, **we intentionally do not inherit environment variables** (apart from one: `MAIN_IMAGE_TAG`). Instead,
all configuration lives in a YAML config file that you pass explicitly. This
makes deployments self-documenting and reproducible.

#### Example: enabling a feature flag

**Before (environment variables):**

```bash
ROX_NETWORK_GRAPH_EXTERNAL_IPS=true ./deploy/deploy.sh
```

**After (roxie CLI):**

```bash
roxie deploy --features ROX_NETWORK_GRAPH_EXTERNAL_IPS
```

The `--features` flag accepts a comma-separated list. Prefix with `-` to disable:

```bash
roxie deploy --features ROX_NETWORK_GRAPH_EXTERNAL_IPS,-ROX_CISA_KEV
```

**Via deploy.sh with roxie under the hood:**

```bash
./deploy/deploy.sh --features ROX_NETWORK_GRAPH_EXTERNAL_IPS
```

### The roxie config file

Instead of juggling `--set` flags, you can write a YAML config file. This is
the recommended approach for anything beyond the simplest deployments. The
config file has two main sections -- `central` and `securedCluster` -- whose
`spec` fields correspond directly to the
[Central](https://docs.openshift.com/acs/operating/manage-central.html) and
[SecuredCluster](https://docs.openshift.com/acs/operating/manage-secured-clusters.html)
CRD specifications. This means you can customize your deployment in any way
the CRDs support.

Here's an example config file (`my-deployment.yml`):

```yaml
roxie:
# Image tag to deploy can also be set in the config file.
version: "4.11.0"

central:
namespace: stackrox
spec:
central:
exposure:
loadBalancer:
# Expose Central via load balancer
enabled: true
# Feature flags and other env vars for Central:
customize:
envVars:
- name: ROX_NETWORK_GRAPH_EXTERNAL_IPS
value: "true"
- name: ROX_BASELINE_GENERATION_DURATION
value: "5m"
# Scanner V4 configuration:
scannerV4:
scannerComponent: Enabled

securedCluster:
namespace: stackrox
spec:
clusterName: my-cluster
# Feature flags for Sensor:
customize:
envVars:
- name: ROX_INIT_CONTAINER_SUPPORT
value: "true"
# Per-node configuration:
perNode:
fileActivityMonitoring:
mode: Enabled
```

Deploy with it:

```bash
roxie deploy --config my-deployment.yml

# Or via deploy.sh:
./deploy/deploy.sh --config my-deployment.yml
```

The `spec` paths mirror the CRDs, so anything you can configure in a
`Central` or `SecuredCluster` custom resource, you can configure here. Check
the CRD documentation or the operator source in `/operator/` for the full
set of available fields.

### Image tag

The roxie flow in `deploy.sh` also supports `MAIN_IMAGE_TAG` for convenience:

```bash
MAIN_IMAGE_TAG=4.11.0 ./deploy/deploy.sh
```

But the recommended approach is the explicit CLI flag:

```bash
roxie deploy --tag 4.11.0
```

### Feedback

We're actively improving roxie and would love your feedback! If you run into
issues, have feature requests, or just want to share your experience, please
reach out to the **ACS Install team**.

---

## Legacy deployment flow

> **Note:** The sections below describe the legacy `kubectl`-based deployment
> flow. Consider switching to roxie (see above) for a better experience.

### Usage

```
# Deploy sripts should be used from the git root of this repo
# Deploy scripts should be used from the git root of this repo
# Deploy StackRox locally on Kubernetes
$ ./deploy/k8s/deploy-local.sh

Expand All @@ -14,7 +190,7 @@ $ ./deploy/openshift/deploy-local.sh
$ LOAD_BALANCER=route ./deploy/openshift/deploy.sh
```

## Env variables
### Env variables

Most environment variables can be found in [common/env.sh](common/env.sh).

Expand Down
60 changes: 60 additions & 0 deletions deploy/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,66 @@
set -e

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)"

if [[ "${USE_ROXIE_DEPLOY:-false}" == "true" ]]; then
ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
if [[ "${SKIP_ROXIE_DEPLOY_BANNER:-false}" != "true" ]]; then
cat <<EOF
----------------------------------------------------------------------------------------
Using roxie to deploy StackRox. 🎉
If this is not intended, please unset USE_ROXIE_DEPLOY and re-run the deployment script.

This is a limited shell wrapper leveraging roxie under the hood.
In particular, you won't be able to configure feature flags by setting environment
variables. Instead, you can pass roxie command line arguments to this script,
for example:

${ROOT}/deploy/deploy.sh --features +ROX_I_WANT_THIS,-ROX_AND_NOT_THIS

roxie will deploy StackRox and then spawn a sub-shell for you to interact with central.
If the shell is closed accidentally, you can use

${ROOT}/deploy/deploy.sh shell

for re-entering the roxie sub-shell.

For tearing down the deployment, invoke:
${ROOT}/scripts/teardown.sh

You can find quick start instructions for roxie in:

${DIR}/README.md.

If you do not want to see this banner again, set

SKIP_ROXIE_DEPLOY_BANNER=true

----------------------------------------------------------------------------------------
EOF
fi
if [[ "${1:-}" == "shell" ]]; then
"${ROOT}/scripts/roxie.sh" shell
else
"${ROOT}/scripts/roxie.sh" --config "${ROOT}/deploy/roxie-config.yaml" deploy "$@"
fi
exit $?
Comment thread
coderabbitai[bot] marked this conversation as resolved.
fi

cat <<EOF
-------------------------------------------------------------------------------------
USE_ROXIE_DEPLOY not enabled, using legacy flow to deploy StackRox
Please consider setting

USE_ROXIE_DEPLOY=true

to opt into roxie-based deployments.

You can find quick start instructions for roxie in:

${DIR}/README.md.
-------------------------------------------------------------------------------------
EOF

# shellcheck source=./detect.sh
source "${DIR}/detect.sh"

Expand Down
6 changes: 6 additions & 0 deletions deploy/roxie-config.yaml
Loading
Loading