start-proxy selects an AMD64 binary on Linux ARM64 · Issue #4173 · github/codeql-action · GitHub
Skip to content

start-proxy selects an AMD64 binary on Linux ARM64 #4173

Closed
Closed
start-proxy selects an AMD64 binary on Linux ARM64#4173

Description

@debanjanbasu
debanjanbasu
opened on Sep 25, 2026

Problem

The CodeQL CLI now has native Linux ARM64 support in beta, but the managed private-registry proxy still selects and downloads an AMD64 executable on Linux ARM64. This prevents the proxy from running natively even when CodeQL analysis itself can run.

This report concerns the proxy artifact and architecture selection, not generic CodeQL CLI ARM64 support, registry credentials, custom CA certificates, or outbound network access.

Source and public release evidence

Source inspected on September 25, 2026: fa8392b7e54a5d74a53270a4aa7defa307aa415c.

  • getProxyPackage() selects by process.platform only. Every Linux host receives update-job-proxy-linux64.tar.gz; process.arch is not checked.
  • getDownloadUrl() looks for that filename in the CodeQL bundle release, then falls back to the hard-coded v2.22.0 release.
  • The v2.27.0 release contains a native codeql-bundle-linux-arm64 CLI bundle, but the proxy assets are only linux64, osx64, and win64.
  • Fresh inspection of both Linux proxy archives below found ELF e_machine=62 (EM_X86_64), not 183 (EM_AARCH64). Downloaded archive hashes match the digests returned by the GitHub Releases API.
Proxy release ELF architecture Archive SHA-256
v2.27.0 AMD64 / e_machine=62 77197411d8fe7c6d157f694e2e4edd6b94bf7d9d10c667864cd6a113c31f97b2
v2.22.0 fallback AMD64 / e_machine=62 caed6d4ad41849570f6c62f784f20f9414a82738b120f464d7fcad1640f7900b

Public-only reproduction

This inspects public artifacts without executing them, extracting them to disk, accessing a private registry, or requiring credentials. It can run on any host with curl and Python 3; the affected execution platform is native Linux ARM64 without x86 emulation.

for version in 2.27.0 2.22.0; do
  curl --fail --silent --show-error --location \
    "https://github.com/github/codeql-action/releases/download/codeql-bundle-v${version}/update-job-proxy-linux64.tar.gz" \
    -o "proxy-v${version}.tar.gz" || exit 1
done

python3 - <<'PY'
import tarfile
from pathlib import Path

for version in ("2.27.0", "2.22.0"):
    with tarfile.open(f"proxy-v{version}.tar.gz", "r:gz") as archive:
        members = [
            member for member in archive.getmembers()
            if member.isfile() and Path(member.name).name == "update-job-proxy"
        ]
        assert len(members) == 1
        header = archive.extractfile(members[0]).read(64)
        assert header[:4] == b"\x7fELF"
        assert header[5] in (1, 2)
        byteorder = "little" if header[5] == 1 else "big"
        machine = int.from_bytes(header[18:20], byteorder)
        print(f"{version}: ELF e_machine={machine} (62=AMD64, 183=AArch64)")
PY

Actual output:

2.27.0: ELF e_machine=62 (62=AMD64, 183=AArch64)
2.22.0: ELF e_machine=62 (62=AMD64, 183=AArch64)

Native-run observation and impact

A September 21, 2026 managed CodeQL run on native Linux ARM64, Ubuntu 26.04.1, with CLI 2.27.0 uploaded analyses successfully, but its local registry-proxy connection tests failed with ECONNREFUSED; proxy cleanup reported ESRCH. This earlier runtime observation is separate from the fresh public-artifact inspection above; no new managed scan was triggered for this report.

Successful analysis upload therefore did not establish successful private-dependency resolution. The architecture mismatch can be verified independently of that runner OS version and of any particular private registry. An x64 runner remains the workaround for this proxy path.

Expected behavior

Select a native Linux ARM64 proxy artifact when process.arch === "arm64", including an architecture-compatible fallback. Until such an artifact is available, fail early with an explicit unsupported-architecture diagnostic rather than attempting to start the AMD64 executable. Tests should cover Linux x64 and arm64 selection and fallback behavior.

Could the maintainers confirm whether private-registry proxy support is intended to be part of Linux ARM64 CodeQL support, or whether this remains an unsupported combination that should be documented explicitly?

Related, but distinct

#3530 and #3531 address proxy startup retries and false-success/readiness reporting. They may explain why a dead proxy is not surfaced clearly, but they do not add an ARM64 proxy artifact or change architecture selection. This report tracks that architecture gap separately.

Activity

  1. mbg commented on Sep 25, 2026

    @mbg
    mbg
    on Sep 25, 2026
    Member

    Hi @debanjanbasu,

    Thanks for reporting this. I can confirm that this isn't supported yet, but this isn't only a limitation in start-proxy's download logic. There are currently no arm64 binaries being built for the proxy, and none are bundled with CodeQL releases, so just changing start-proxy won't resolve this. We will look into getting this resolved.

  2. mbg commented on Sep 29, 2026

    @mbg
    mbg
    on Sep 29, 2026
    Member

    To follow-up on this, we have made all the necessary changes to allow start-proxy to work on linux-arm64. Because of how the binaries are distributed, this will require the next CodeQL CLI release and associated CodeQL Action release to take place before these changes will be available. Both are currently expected for next week.

  3. debanjanbasu commented on Oct 6, 2026

    @debanjanbasu
    debanjanbasu
    on Oct 6, 2026
    Author

    Thanks @mbg — great to see #4181 merged. One additional detail for GitHub Enterprise Cloud with data residency (GHE.com) that may stop linux-arm64 working there even after the next CLI/Action release:

    What happens on GHE.com today (observed 2026-10-06 with codeql-action@v4.38.2, CLI 2.27.1, x64 runner, default setup with a private registry):

    Setup proxy for registries:
  ##[warning]Failed to retrieve information about the linked release: Not Found - https://docs.github.com/rest/releases/releases#get-a-release-by-tag-name
  Did not find 'update-job-proxy-linux64.tar.gz' in the linked release, falling back to hard-coded version.
Initialize CodeQL:
  Looked for CodeQL bundle codeql-bundle-linux64.tar.zst in github/codeql-action on https://<tenant>.ghe.com but got error HttpError: Not Found …
    • getReleaseByVersion() calls getApiClient().rest.repos.getReleaseByTag({owner: "github", repo: "codeql-action", …}) against the tenant API. github/codeql-action does not exist on a GHE.com tenant, so this always 404s there.
    • getDownloadUrl() then uses getFallbackUrl(), which on main (after Support linux-arm64 assets in start-proxy #4181) still points at UPDATEJOB_PROXY_URL_PREFIX = …/releases/download/codeql-bundle-v2.22.0/. That release has no update-job-proxy-linux-arm64.tar.gz, so on GHE.com an ARM64 runner would still be handed a missing (or AMD64) asset regardless of the new release.
    • The CLI bundle path behaves differently: after the same tenant 404, init continues and obtains the bundle from github.com. The proxy path has no equivalent github.com lookup — only the hard-coded fallback.

    Possible fixes (either would do):

    1. When the tenant lookup 404s on GHE.com/GHES, look up the linked release on github.com (as the bundle download effectively does), or
    2. Move the hard-coded fallback to a bundle release that ships update-job-proxy-linux-arm64.tar.gz once one exists (and keep it current).

    Happy to re-test on GHE.com with ARM64 runners as soon as the release is out.

  4. debanjanbasu commented on Oct 10, 2026

    @debanjanbasu
    debanjanbasu
    on Oct 10, 2026
    Author

    Fresh native Linux ARM64 validation on GitHub Enterprise Cloud with data residency (GHE.com), 2026-10-10, using released v4.38.3 and bundle codeql-bundle-v2.27.2:

    • Runner reports aarch64.
    • A separate anonymous download of update-job-proxy-linux-arm64.tar.gz from the 2.27.2 public release succeeded and matched the published SHA-256. readelf identifies Machine: AArch64. This binary was not installed in the action toolcache.
    • The unmodified SHA-pinned start-proxy action, using the built-in workflow token with its documented permission and a registry entry, then logged:
    Failed to retrieve information about the linked release: Not Found
Did not find 'update-job-proxy-linux-arm64.tar.gz' in the linked release, falling back to hard-coded version.
start-proxy action failed: Unexpected HTTP response: 404

    So #4181 and the new asset address architecture selection, but the released GHE.com resolution path still fails before proxy startup. This was a standalone action diagnostic, not a successful default-setup scan or private dependency restore. Separate CodeQL default-setup jobs on x64 successfully used CLI 2.27.2 the same day.

    Does #4198 cover this released failure, and can you confirm the target release or a supported GHE.com default-setup workaround? We can re-test after a release. No fleet routing was changed.

  5. mbg commented on Oct 16, 2026

    @mbg
    mbg
    on Oct 16, 2026
    Member

    Hi @debanjanbasu,

    Yes, #4198 is intended to address the issue you flagged in #4173 (comment). This was now released yesterday as part of CodeQL Action v4.39.0/v3.39.0. It would be great if you could confirm whether this resolves the issue for you or whether you are still seeing the start-proxy step fallback to an outdated version?

  6. debanjanbasu commented on Oct 18, 2026

    @debanjanbasu
    debanjanbasu
    on Oct 18, 2026
    Author

    Thanks @mbg, confirmed: v4.39.0 (#4198) fixes this for us on GitHub Enterprise Cloud with data residency (GHE.com).

    • x64 default setup (2026-10-16, v4.39.0): start-proxy still logs the tenant 404 for the linked-release lookup. It then logs Using authentication proxy from CodeQL release 2.27.2, downloads the 2.27.2 proxy from github.com, starts it and reaches our private registry. It no longer falls back to the 2.22.0 hard-coded version.
    • Native Linux ARM64 (2026-10-16, the unmodified SHA-pinned v4.39.0 start-proxy with the built-in workflow token): the action installed update-job-proxy 2.27.2 for arm64 (file reports an AArch64 ELF), logged Proxy started on 127.0.0.1:… and connected to the registry successfully.

    So the architecture selection (#4181) and the GHE.com release resolution (#4198) both work for us now. In a later ARM64 default-setup trial, private-registry connections failed only because of our own self-hosted runner image's trust configuration, not because of the action. From our side this issue can be closed.

  7. mbg commented on Oct 19, 2026

    @mbg
    mbg
    on Oct 19, 2026
    Member

    Great, thanks for confirming @debanjanbasu!

  8. mbg
    closed this as completedon Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions