fix: strip the token already embedded in remote.origin.url before Repository.push by MincongZhou · Pull Request #1815 · modelscope/modelscope · GitHub
Skip to content

fix: strip the token already embedded in remote.origin.url before Repository.push - #1815

Open
MincongZhou wants to merge 2 commits into
modelscope:masterfrom
MincongZhou:fix/repository-push-strip-stale-token
Open

MincongZhou wants to merge 2 commits into
modelscope:masterfrom
MincongZhou:fix/repository-push-strip-stale-token

Conversation

@MincongZhou

Copy link
Copy Markdown

What this fixes

Repository.push() hands the URL it reads back from the checkout straight to git push:

# modelscope/hub/repository.py:141
url = self.git_wrapper.get_repo_remote_url(self.model_dir)
...
self.git_wrapper.push(
    repo_dir=self.model_dir,
    git_token=self.git_token,     # the caller's fresh token
    url=url,                      # ... but this URL still carries the old one
    ...
)

A URL obtained from an existing checkout still contains the token that was embedded when the repository was cloned. config_git_token returns early when the remote already contains //oauth2, and _inject_token likewise passes an already-tokened URL through unchanged — so self.git_token is silently discarded and git authenticates with the stale token.

Its twin does not have this problem. DatasetRepository.push() strips first, and so does _clone_if_needed:

# :253-255  (DatasetRepository.push)
remote_url = self.git_wrapper.get_repo_remote_url(self.repo_work_dir)
remote_url = self.git_wrapper.remove_token_from_url(remote_url)

# :45-46  (_clone_if_needed)
existing = git_wrapper.get_repo_remote_url(repo_dir)
existing = git_wrapper.remove_token_from_url(existing)

Repository.push() is the only one of the three call sites that omits it.

Consequence

After a token rotation, a push from an existing checkout fails authentication even though the caller passed a valid git_token, and there is no SDK-level recovery — the stale secret stays in .git/config. DatasetRepository.push() with the same token on the same machine works, which makes it look like a permissions problem rather than a stale-credential one. save_pretrained(push_to_hub=True) (patcher.py:487), cli/modelcard.py:141 and push_to_hub.py:66 all route through Repository.push().

Reproduction

Both push methods were executed from the real source (function AST extracted and bound to a stub object, with a recorder git_wrapper; only the network-facing wrapper is stubbed). The remote URL the wrapper returns is tokened, exactly as an existing clone would be, and both callers pass git_token="NEW_TOKEN". The recorder prints the URL that would reach git push:

BEFORE (upstream main)
  Repository          git_token=NEW_TOKEN   url userinfo = OLD_SECRET    <-- the stale token is what git uses
  DatasetRepository   git_token=NEW_TOKEN   url userinfo = (no embedded token)   ok

AFTER (this change)
  Repository          git_token=NEW_TOKEN   url userinfo = (no embedded token)   ok
  DatasetRepository   git_token=NEW_TOKEN   url userinfo = (no embedded token)   ok

DatasetRepository is unchanged in both runs; only Repository changes.

Change

One line:

         url = self.git_wrapper.get_repo_remote_url(self.model_dir)
+        url = self.git_wrapper.remove_token_from_url(url)

Validation

  • python -m py_compile modelscope/hub/repository.py → OK
  • flake8 --max-line-length=120 modelscope/hub/repository.py → exit 0
  • yapf --diff (the hook the repo pins at v0.30.0) → no diff
  • The reproduction above, before and after.

Not verified: a real push against the Hub — I have no credentials for it. The claim is limited to what the method passes to the git wrapper, which is where the divergence is. I also did not run the repository's own test suite (it is not installed in this environment).

Notes for reviewers

  • Repository.push() carries a DeprecationWarning pointing at HubApi().upload_folder. I fixed it rather than leaving it, because the documented successor does not automatically cover the callers that still route through it (save_pretrained(push_to_hub=True), cli/modelcard.py, push_to_hub.py) and because the fix is what makes it agree with its twin. Happy to close this in favour of a different approach if you would rather deprecate harder.
  • The missing strip is long-standing rather than a recent regression, so there is no issue to reference.

Competition

No competing item. gh searches for remove_token_from_url, Repository.push and git_token return no matching issue or PR, and none of the six currently open PRs touches modelscope/hub/repository.py (the merged #1592 touched only hub/api.py).

…ository.push

Repository.push hands the URL read back from the checkout straight to git push:

    url = self.git_wrapper.get_repo_remote_url(self.model_dir)   # :141
    ...
    self.git_wrapper.push(..., git_token=self.git_token, url=url)

A URL taken from an existing checkout still carries the token that was used at
clone time. config_git_token returns early when the remote already contains
'//oauth2', and _inject_token likewise passes an already-tokened URL through,
so the freshly supplied git_token is silently dropped and git authenticates
with the stale one.

Its twin does not have this problem -- DatasetRepository.push strips first:

    remote_url = self.git_wrapper.remove_token_from_url(remote_url)   # :255

as does _clone_if_needed at :46. Repository.push is the only one of the three
call sites that omits it, so after a token rotation a push from an existing
checkout fails authentication even though the caller passed a valid token, and
the stale secret stays in .git/config. save_pretrained(push_to_hub=True),
cli/modelcard.py and push_to_hub.py all route through here.

Strip the token for Repository.push as well, leaving git_token as the only
credential used.
Repository.push and DatasetRepository.push both have to hand the git wrapper a
URL with no token embedded in it: the URL read back from an existing checkout
still carries the clone-time token, and config_git_token/_inject_token leave an
already-tokened URL untouched, so the embedded token silently beats the
git_token the caller just supplied.

The mock git wrapper returns a tokened URL, which is what an existing checkout
provides, and the tests assert the URL that reaches git push - for Repository,
for DatasetRepository, and that the two agree. They also require the strip to go
through remove_token_from_url rather than string surgery at the call site. No
checkout and no network access needed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant