meta: give Db2 containers enough time to initialize before giving up by WikiRik · Pull Request #18341 · sequelize/sequelize · GitHub
Skip to content

meta: give Db2 containers enough time to initialize before giving up - #18341

Draft
WikiRik wants to merge 2 commits into
mainfrom
meta/db2-startup-timeout
Draft

meta: give Db2 containers enough time to initialize before giving up#18341
WikiRik wants to merge 2 commits into
mainfrom
meta/db2-startup-timeout

Conversation

@WikiRik

@WikiRik WikiRik commented Sep 4, 2026

Copy link
Copy Markdown
Member

Pull Request Checklist

  • Have you added new tests to prevent regressions? (N/A - dev tooling)
  • Does yarn test or yarn test-DIALECT pass with this change (including linting)?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)? (N/A)
  • Did you follow the commit message conventions explained in CONTRIBUTING.md?

Description Of Change

Since #18291 landed, db2 oldest and db2 latest are the most frequent database start failures in CI: 15 failed yarn start-db2-* steps across 9 runs, all with

Timeout of 150s exceeded when waiting for container to be healthy: sequelize-db2-oldest

Examples: db2 oldest (Node 22), db2 latest (Node 24).

Cause

Db2 creates the database on first start. The start step on successful main runs takes 127-213s, and locally on a fast machine the container first reports healthy after 175s. The Db2-only copy of wait-until-healthy.sh polls for 10 × 15s = 150s, which is right at the edge of a normal startup, so a slightly slow runner fails.

That copy also predates #18291: it still has the dead return_code check, prints nothing about the container on failure (which is why the job logs above say nothing useful), and sleeps a blind 15s after the container is healthy.

There is a second, latent problem. The image's setup script restarts the instance twice right after CREATE DATABASE:

21:45:21  SQL1026N  The database manager is already active.   <- db created, connectable
21:45:27  SQL1064N  DB2STOP processing was successful.
21:45:28  SQL1063N  DB2START processing was successful.
21:45:34  SQL1064N  DB2STOP processing was successful.
21:45:36  SQL1063N  DB2START processing was successful.
21:45:36  (*) Setup has completed.

A probe landing in one of those ~6s windows marks the container healthy, after which the instance goes down again and check-connection.ts gets ECONNREFUSED. The old sleep 15 was presumably guarding against this.

Changes

  • Delete dev/db2/wait-until-healthy.sh; both start.sh use the shared dev/wait-until-healthy.sh with HEALTHCHECK_TIMEOUT=600, so a slow runner has headroom while a genuinely stuck container still fails in minutes rather than hours, and the health probes + container logs are dumped on failure.
  • The healthcheck additionally requires /database/config/.shared-data/setup_complete, the marker the image's setup_db2_instance.sh touches right after the last restart (same path in 11.5.5.1 and 12.1.4.0). This replaces the blind sleep.
  • start_period: 300s so the container is not reported unhealthy while it is legitimately still creating the database.

Verification (local, icr.io/db2_community/db2:11.5.5.1)

Healthcheck command exit
after setup completed 0
marker missing 1
marker present, database unreachable 1

Note: touches the same start.sh files as #18340, expect a trivial conflict in whichever lands last.

🤖 Generated with Claude Code

db2 oldest/latest jobs fail regularly in `yarn start-db2-*` with
"Timeout of 150s exceeded when waiting for container to be healthy".
Db2 creates the database on first start, and successful runs of the
same step on main take 127-213s, so the 150s budget of the Db2-only
copy of wait-until-healthy.sh is right at the edge of normal. Locally,
on a fast machine, the container first reports healthy after 175s.

That copy also predates #18291: it still has the dead `return_code`
check, prints nothing about the container on failure, and sleeps a
blind 15s after the container is healthy.

Remove it and use the shared dev/wait-until-healthy.sh, which
supports HEALTHCHECK_TIMEOUT and dumps the health probes and container
logs when it gives up. Db2 gets a 600s budget so a slow runner has
headroom while a genuinely stuck container still fails in minutes
rather than hours.

The setup script restarts the instance twice after creating the
database, so a probe can pass in a short window before a restart.
The healthcheck now also requires the image's own setup-complete
marker, which is created after the last restart. This replaces the
blind 15s sleep. start_period goes to 300s so the container is not
reported unhealthy while it is legitimately still creating the
database.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant