{{ message }}
Repository navigation
feat: add GitLab and Bitbucket to --source - #1356
Merged
Merged
Conversation
Extends `apify create --source github` to the other two providers the API is gaining in apify-core#30208. The API addresses GitHub by a fixed `github-app` id, but GitLab and Bitbucket by the id of each connected account — a user can connect several, and each carries its own workspaces. So the CLI now matches integrations by `provider`, merges the workspaces of every account into one list, and keeps the owning account on each entry as the id later calls address it by. `--git-repo` matches a workspace label as well as its id, because GitLab identifies a namespace by a number nobody would type. Connecting is the part the CLI cannot do alone: GitLab and Bitbucket mint their CSRF state server-side, so `connectViaConsole` hands off to the Console page from apify-core#30279 and waits on a loopback server, the same way `apify login` does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- parse `--git-repo` at the last slash so GitLab subgroup workspaces work
- use provider labels ("GitLab") instead of raw ids in user-facing messages
- generalize `--source` help and `--git-repo` error text to all providers
- reuse `getConsoleIntegrationsUrl()` in `apify login`
- type `connectViaConsole` provider and `stopReason`
- add tests for the Console hand-off and `readProviderState`
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found by running the GitLab and Bitbucket flows against the live API. - poll the integrations listing alongside the Console callback, so a connection made on the page by hand is picked up instead of hanging until the timeout; `AbortSignal` releases whichever wait loses - keep the Actor and its deploy key when the clone fails, and report the missing local copy with a recoverable next step - strip the account name Bitbucket embeds in its clone URL, which made git ask only for a password and no token could satisfy - report git's own stderr instead of execa's wrapper, offer SSH beside HTTPS when the clone has to be redone, and stop leading with `cd` into a directory the clone never filled Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Master's #1371 and #1373 rewrote the same functions this branch generalizes. Resolved by keeping this branch's multi-provider shape and folding both of master's behaviours into it: - `getGitConnectUrl` keeps the optional `authorize` block, so GitLab and Bitbucket still fall back to the Console page, and now also sends `routePrefix` for an organization login - `ensureUsableIntegration` keeps the Console hand-off race for providers the CLI cannot authorize, and takes master's per-poll re-read of which grant is missing, its never-open-the-same-URL-twice rule, and the account-aware wait message — with provider labels rather than ids - `runGitSourceFlow` keeps the non-fatal clone and per-account `providerId`, and takes master's re-prompt for a rejected repository name - `notAuthorized` and `noWorkspace` name the Apify account and the provider label; next steps and `getGitStopUrl` carry the account - both test suites kept in full Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…aces Both found by reviewing the Console hand-off race. - a failing integrations read no longer ends the wait: `pollForWorkspaces` keeps polling, so one 5xx or a dropped connection cannot turn a grant that is still coming into a lookup failure - the race aborts in a `finally`. The abort is what closes the loopback server and clears its timer, so skipping it held `apify create` open for the full five minutes after the command had already printed its next steps - a `connected: true` callback no longer stops on one empty read. Console reports the connection as soon as the integration exists, which can beat its workspaces into the listing, so the flow reported a connected account as a missing one. It now waits the way the authorize branch does - both cases covered by tests, and the suite's real 2 s sleep replaced with an abort-faithful `node:timers/promises` mock: 2.19 s to 37 ms Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#1371 fixed this for GitHub. These two providers reach Console a different way — the CLI navigates to the integrations page rather than sending a `routePrefix` through an OAuth callback — so the same bug is still there. Console creates the `UserIntegration` for the account its own route names (`useEffectiveUserSession` reads `viewedUserId` from the path), and where the path says nothing it redirects to whichever account the browser was last on: `/settings/*` is `ROUTE_KEYS.ACCOUNT`, which is in Console's `REMEMBER_LAST_ACCOUNT_ROUTE_NAMES`. `GET /integrations/git` is scoped to the token owner, so an integration that lands elsewhere is invisible to the CLI, which then waits out the full five minutes. That makes it fail in both directions: an organization login connecting to the personal account, and a personal login connecting to an organization the browser was last on. GitHub only ever had the first. The account now rides in the path. An explicit `/organization/<id>` route is never redirected away, so it pins both cases. - `getConsoleRoutePrefix` moves to `console-url.ts` beside `getConsoleIntegrationsUrl`, which takes the account. GitHub's query param and these providers' path prefix now share one source - the account reaches the page the CLI opens, the connect URL printed for agents and CI, and the `noWorkspace` next steps - `connectViaConsole` takes `{ abortSignal, account }` rather than two positional optionals Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
l2ysho
commented
Sep 1, 2026
Contributor
Author
There was a problem hiding this comment.
This is idea from https://github.com/apify/apify-core/pull/30279 but when I see it now, just poll GET integration/git seems to be enough and /exit is not so big deal (GitHub does not have it anyway) 🤔
Contributor
There was a problem hiding this comment.
Relaying on the loopback server request from Console seems a bit cleaner solution to me than polling API, honestly.
The exit for the Gitlab and Bitbucket is also beneficial. Once we convert Github to UserIntegration, we will have to implement deeplink for the GH as well.
l2ysho
marked this pull request as ready for review
September 1, 2026 16:54
DaveHanns
approved these changes
Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Note
TL;DR — adds
--source gitlaband--source bitbucketalongsidegithub. Both tested with personal and repo token.Follows #1348. Part of #1153.
No longer blocked
/integrations/git,create-repoandregister-deployment-keyThe shape change
GitHub is one connection addressed by the fixed id
github-app. GitLab and Bitbucket are aUserIntegrationper connected account, each addressed by its own id and reporting its own workspaces — one GitLab account can carry several namespaces, including subgroups. So:provider, not by a hardcoded idproviderIdthatcreate-repoandregister-deployment-keyaddress it byWithout that, a second connected account would have been silently ignored.
--git-repomatches labelsGitLab identifies a namespace by a number, so
--git-repo 4711/my-scraperis what the API wants and nobody would type it.--git-repo my-group/subresolves through the label. Since a GitLab label is afull_pathand contains slashes of its own, the flag splits at the last slash —my-group/sub/my-scraperis workspacemy-group/sub, repositorymy-scraper. Theworkspacesarray in--jsonreports labels for the same reason.Connecting
GitHub's authorize URL is two public query params, so the CLI builds it. GitLab and Bitbucket mint a CSRF
stateserver-side and need aUserIntegrationrow to exist first — neither is something the CLI can produce. SoconnectViaConsoleopens the Console integrations page with thelocalCli*deep-link params and waits on a loopback server, exactly asapify logindoes.The callback is not treated as the only signal. The CLI polls
GET /integrations/gitalongside it and takes whichever lands first, because a user who connects on the Console page by hand never triggers the callback — and that is a real path, not a hypothetical: it is what happens when the hand-off does not fire. AnAbortSignalreleases whichever wait loses, so the loopback server never holds the command open.Which Apify account the connection lands on
#1371 fixed this for GitHub. These two providers reach Console a different way — the CLI navigates to the integrations page rather than sending a
routePrefixthrough an OAuth callback — so the bug survived.Console creates the
UserIntegrationfor the account its own route names:useEffectiveUserSessionresolves the session token fromviewedUserId, which comes only from the path. Where the path says nothing, Console redirects to whichever account the browser was last on —/settings/*isROUTE_KEYS.ACCOUNT, which is inREMEMBER_LAST_ACCOUNT_ROUTE_NAMES.GET /integrations/gitis scoped to the token owner, so an integration created elsewhere is invisible to the CLI, which then waits out its full timeout and reports the wrong cause.It fails in both directions, unlike GitHub:
The account now rides in the path. An explicit
/organization/<id>route is never redirected away (if (!isUserRoute) returninuseRedirectToLastAccount), so it pins both cases.getConsoleRoutePrefixmoved toconsole-url.tsbesidegetConsoleIntegrationsUrl, which now takes the account — GitHub's query param and these providers' path prefix share one source. The account reaches the page the CLI opens, the connect URL printed for agents and CI, and thenoWorkspacenext steps.Waiting for the connection
Two defects in the race, both found in review:
pollForWorkspaceshad no guard aroundload(), so one 5xx or a dropped connection rejected out of the race and skippedcontroller.abort()— the only thing that closes the loopback server and clears its timer.apify createprinted its next steps and then sat there for the remaining five minutes. The poll now survives a failed read, and the abort runs in afinally.A failed clone no longer costs the Actor
The clone is the only step needing the user's own provider credentials, and the only purely local one. It used to abort the run, leaving a repository with no Actor and a name that the next attempt could not reuse.
Now the error is held, the Actor and the deploy key are still created, and the stop reports the missing local copy with a
git cloneto run — over HTTPS or SSH, whichever the user has credentials for. Bitbucket also returns its clone URL with the account name embedded, which makes git ask only for a password and defeats every token; the CLI strips the userinfo before cloning or printing it.Reviewer notes
This branch depends on Console's routing, and none of it is a contract. Three assumptions, all in apify-core, none covered by a test that would fail if someone changed them:
/organization/:viewedUserIdtwin (route_definitions.ts)if (!isUserRoute) return)useEffectiveUserSession)A regression in any of them is silent — no error, no 404, the connection just lands on the wrong account and the user is told Apify has no GitLab account. Cheapest mitigation is a test on the apify-core side; the real one is the first follow-up below.
GitHub's
providervalue is confirmed, not assumed.src/api/src/routes/integrations/git_providers.tsbuilds the GitHub entry as{ id: GITHUB_APP_PROVIDER_ID, provider: 'github' }withGITHUB_APP_PROVIDER_ID = 'github-app', so matching byprovideris correct for all three.Verification
Verified live against production, on a personal login:
The account routing and the two waiting fixes landed after those runs and are covered by unit tests only — an organization login has not been exercised end to end.
64 tests across
gitSource.test.tsandconnectViaConsole.test.ts. Full suite: 527 passed, 4 skipped.lint,format,buildclean,docs/reference.mdregenerated.Follow-ups, not in this PR
/exitsignal, which tells the CLI the user closed the tab; GitHub has never had it, so this is parity rather than a regression. Losing it means a user who gives up waits out the timeout.POST /v2/integrations/gitreturning{ integrationId, loginUrl }would let the CLI start the flow under its own token. The account question then never gets asked, and all three Console-routing assumptions above go away.create-repomaps to "Reconnect your account", which cannot help when the cause is missing permission; andshouldValidateScopes: falselets a Bitbucket token withoutrepository:adminconnect cleanly and fail three steps later.connectViaConsole.tsandauth/login.ts. Worth extracting next tolocal-api-server.ts, which also touchesedit-input-schema.ts. Note the doubleencodeURIComponentonlocalCliComputerNamein both —searchParams.setalready encodes, so whoever extracts this has to keep the quirk or fix both sides at once.🤖 Generated with Claude Code