fix(ci): isolate HTTP end-to-end suites and stop the SCIM readiness flake by waleedlatif1 · Pull Request #8544 · simstudioai/sim · GitHub
Skip to content

fix(ci): isolate HTTP end-to-end suites and stop the SCIM readiness flake - #8544

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/ci-scim-flake
Oct 1, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/ci-scim-flake

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Flake root cause: the SCIM step starts next dev and waits for /api/health, whose first request cold-compiles the app under Turbopack. Across 40 recent green runs readiness took 42–93s (step totals up to 209s), and today's staging push hit >120s — the fixed 120s deadline failed on the slow tail. The server log was never uploaded, so failures were undiagnosable.
  • New End-to-end over real HTTP job holds SCIM + the four search HTTP suites, on its own Postgres (provisioned via db:migrate, the production path). Previously SCIM ran on both integration matrix legs, which are the PR critical path (~13 min avg vs 7.6 min Lint and Test); the search suites were gated by matrix.provision == 'push'. Isolation also matters: the SCIM app boots hosted, which starts background usage replay against DATABASE_URL.
  • Readiness: 300s deadline (hang catcher only), step timeout-minutes: 12, fail-fast on server exit, server log tail printed inline, and the full scim-next.log uploaded with the reports (one artifact instead of five upload steps).
  • setup-workspace composite action replaces four copies of Bun/Node/cache/install setup. Cache keys are unchanged (existing sticky disks are reused); the integration legs now also mount the lockfile-keyed node_modules disk.
  • Removed a duplicated comment paragraph in the base-ref step.

Expected effect: ~2 min off each integration leg (the PR critical path), SCIM run once instead of twice, and the readiness flake gone.

ci-score (StarSling) rates the repo 91/100 before and after; its one failing check (fetch-depth: 0 in helm.yml) is a false positive — ct diffs against the PR base.

Test plan

  • actionlint — no new findings
  • CI on this PR: new End-to-end over real HTTP job green; integration legs no longer run SCIM

…lake

The SCIM step cold-compiles the app under next dev before its first
request; readiness took 42-150s on 8 vCPU runners against a fixed 120s
deadline, so the slow tail failed. It also ran on both provisioning legs
of the integration matrix, which are the PR critical path (~13 min).

- Move SCIM and the four search HTTP suites into their own job with its
  own database, off the integration legs and run once against migrate.
- Readiness waits up to 300s, fails fast if the server exits, prints the
  server log tail, and uploads the server log with the reports.
- Factor Bun/Node/cache/install setup into a setup-workspace action.
@vercel

vercel Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Comment thread .github/workflows/test-build.yml Outdated
Comment thread .github/workflows/test-build.yml
@waleedlatif1
waleedlatif1 merged commit efca99c into staging Oct 1, 2026
31 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/ci-scim-flake branch October 1, 2026 21:48

This branch was successfully deployed

1 active deployment
Preview — 1c352c9a Deployed Oct 1, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant