{{ message }}
Repository navigation
ci(ric): run smoke and OS integration tests on CodeBuild runners - #666
Open
darklight3it wants to merge 14 commits into
Open
darklight3it wants to merge 14 commits into
darklight3it wants to merge 14 commits into
Conversation
The RIC smoke and OS integration tests ran their buildspecs through CodeBuild local on GitHub-hosted runners. Every shard pulled the 1.3 GB codebuild/local-builds image plus all base images anonymously from public.ecr.aws, and GitHub runner IPs are shared with other tenants, so the checks kept failing on its anonymous data limit. - Replace CodeBuild local with run_os_test.sh, one script that runs one OS / distro version / runtime / platform test directly on the host. The per-OS buildspecs, the Makefile targets and CI all call it, so the same tests run locally again (`make test-integ`, `make test-smoke-<arch>`). - Move the CI matrix to test/integration/os-test-matrix.json, shared by the workflow and `make test-integ` (run_all_os_tests.sh). - Run the smoke and integration jobs on CodeBuild-hosted runners and pull base images (function images and the JNI build) through an ECR pull-through cache via BASE_REGISTRY. - Skip these jobs on fork PRs, since the runners run in our AWS account; their checks pass with a notice and a maintainer runs the tests from a branch in this repository before merging. - Install a qemu emulator only when a platform differs from the host, and use the long invocation timeout for any emulated platform. - Fix run_invocation_test.sh referencing the unset RUNTIME variable.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #666 +/- ##
=========================================
Coverage 66.66% 66.66%
Complexity 223 223
=========================================
Files 35 35
Lines 1020 1020
Branches 146 146
=========================================
Hits 680 680
Misses 288 288
Partials 52 52 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Run the smoke and integration jobs on the existing release CodeBuild runners (aws-lambda-java-libs-test-trigger-<arch>) instead of dedicated ones. Those runners are egress-locked, so do what the release jobs do: use the image's preinstalled Corretto 8 (new use-runner-corretto8 action) instead of actions/setup-java, and resolve Maven through the CodeArtifact proxy (configure-maven-mirror). Only the distro package mirrors the test images install from still need egress.
Keep comments to current behaviour; the rationale is in the PR description.
- Fold the smoke tests into the integration matrix: alpine 3.15 was already in it, so only amazoncorretto 11 is added. Removes the smoke-test jobs and the test-smoke / pr-<arch> make targets. - Stop skipping fork PRs. Fork workflows already need maintainer approval to run, which is the actual gate. - Run local integration tests for the host's platform only, which removes the qemu setup from run_os_test.sh.
The JSON matrix only existed so make test-integ could run every combination locally, and local runs are now one combination at a time. Inline the matrix and drop os-test-matrix.json, the load job and run_all_os_tests.sh.
… macOS run_invocation_test.sh only gave emulated runs the long timeout on aarch64 hosts, so emulated tests on macOS (where arch prints arm64 or i386) or on x86_64 hosts used the 10s timeout. Use it for any platform that isn't the host's.
The JNI build pulls amazonlinux:2 and alpine:3 from public.ecr.aws on every Maven build, so the build, coverage and dockerized-test jobs kept failing on the anonymous data limit shared by GitHub runner IPs. Load the images from the Actions cache into the local image store (pulling only on a weekly cache miss) and use the buildx docker driver, which builds from the local store.
run_os_test.sh copied each Dockerfile.function.* into .scratch and appended the curl install and the RIE COPY. Do both in the Dockerfiles (ADD unpacks the RIE archive, RIE is a build arg), so each Dockerfile is the full image definition and the script builds it directly.
- Log in to the ECR pull-through cache with the runner's account and region instead of repository secrets, which fork PRs don't get. - Skip unit tests when building in run_os_test.sh; the build job runs them. - Don't retry the function image build; pulls go through the cache, and a broken Dockerfile should fail fast. - Fold run_invocation_test.sh, fetch_test_container_logs.sh and clean_up.sh into run_os_test.sh, with container and network names per OS / version / runtime so local runs don't collide. - Start the function with `java` from the image's PATH, dropping JAVA_BINARY_LOCATION. - cache-jni-base-images: pull when the cached tar can't be loaded. - dockerized-test: also run when cache-jni-base-images changes.
RIE doesn't look the bootstrap up on PATH, so resolve it in the container's shell.
Every ARG used in FROM needs a default, otherwise BuildKit warns InvalidDefaultArgInFrom. run_os_test.sh always passes them.
darklight3it
marked this pull request as ready for review
October 21, 2026 13:36
antoniaasze
approved these changes
Oct 21, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
Problem
The RIC PR checks keep failing with
429/Data limit exceededfrom public.ecr.aws.codebuild/local-buildsagent image (about 80% of our bytes), then the base images for the JNI build and the function images.Approach
Run the jobs that build and run containers on the existing release CodeBuild runners (
codebuild-aws-lambda-java-libs-test-trigger-<x86|arm64>), the same onesrelease-runtime-interface-client.ymluses. Base images come through an ECR pull-through cache, and CodeBuild local is no longer needed.The release runners are in an egress-locked network, so the jobs do what the release jobs do:
actions/setup-javacan't download one there);configure-maven-mirror).The tests themselves are the same OS / distro version / runtime / platform combinations.
Changes
One test script instead of CodeBuild local
test/integration/codebuild/scripts/run_os_test.shruns one combination on the host. It builds the RIC and the test handler (without unit tests, which thebuildcheck runs), builds the function image with the Runtime Interface Emulator, invokes it and checks the response, then collects container logs and cleans up. It replacesrun_invocation_test.sh,fetch_test_container_logs.shandclean_up.sh. Container names include the OS, version and runtime, so different combinations can run side by side.Dockerfile.function.*is now the full test image: it installs curl (used by the tester container) and unpacks the Runtime Interface Emulator itself, instead of the script appending both to a temporary copy.batchmatrices are unchanged.test/integration/codebuild-local/.docker-retry.shmoved tocodebuild/scripts/.Matrix
Workflow (
runtime-interface-client_pr.yml)integration-test-matrixruns on the release runners, every platform on a native host. Thesmoke-test-archandsmoke-testjobs are removed..github/actions/use-runner-corretto8sets up the image's Corretto 8 and the JDK 8 Maven toolchain.BASE_REGISTRY. The JNI build already reads it, and the function Dockerfiles now take it as a build arg too, defaulting topublic.ecr.aws.max-parallel: 8on the integration matrix, to stay inside the CodeBuild concurrency quota.integration-testandbuildcheck names are unchanged.smoke-testno longer exists, so it must be removed from the required checks.Fork PRs
Local runs
make test-integ OS_DISTRIBUTION=debian DISTRO_VERSION=bookworm RUNTIME_VERSION=corretto11 PLATFORM=linux/amd64runs one combination.make prruns the unit tests. Thetest-smoke-*andpr-<arch>targets are removed.JNI build base images
amazonlinux:2andalpine:3, sobuild,dockerized-testand thecoveragejobs onmainpulled them anonymously on every run, and failed the same way..github/actions/cache-jni-base-imagesloads both images into the local Docker image store from the Actions cache, pulling only on a weekly cache miss. Those jobs use the buildxdockerdriver, which builds from the local store. Works in forks too, with no AWS access.Fixes (carried over into
run_os_test.sh)${RUNTIME}underset -u).aarch64hosts, so emulated tests on macOS (wherearchprintsarm64ori386) or on x86_64 hosts timed out after 10 s. It now applies to any platform that isn't the host's.Coverage and savings
No test coverage is removed.
build, and again inside every integration shardbuildonlybuildbuild(unchanged)Anonymous public.ecr.aws pulls per PR (counted as in #663, one per
docker buildordocker pull)That also drops the 1.3 GB
codebuild/local-buildsagent image every integration and smoke job pulled, about 17 GB per PR across 13 jobs.dockerized-teststill pullslambda/java:25once (see Not in this PR).Time, measured on this repo's runs:
buildjobmax-parallel: 4)max-parallel: 8)The "after" integration times include the full RIC build. Before: the wall clock is attempt 1 of that run (alpine 3.15 and debian x64 failed with
429and were re-run); job times and total are from the final, green jobs. After: most of the ~22 min was spent waiting for CodeBuild runners. All 17 matrix jobs were queued at the start, but the first one got a runner only after ~8 min, and the others were picked up one at a time over the next ~7.5 min. In attempt 1 of the same run (which failed on the network allowlist), runners picked up the jobs within a minute and the whole run took ~10 min.Before merging
smoke-testfrom the required status checks.Not in this PR
test/integration/codebuild/buildspec.os.*.yml) stay for now: the PR workflow no longer uses them, but they still define their ownbatchmatrix for other CodeBuild-based CI. A follow-up PR removes them, along withconfigure_multi_arch_env.sh, once nothing else runs them. Until then, the matrix in the workflow is the one the PR checks use.dockerized-teststill pullslambda/java:25anonymously for its test image.publishjob onmainis removed separately in ci(ric): remove dead snapshot publish job from merge-to-main #665.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.