{{ message }}
Crate: add default strategy user agent - #23832
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The updated fetched-content test expects one page_content call but triggers two.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Sets the Crate livecheck strategy’s default user agent to :browser so crates.io API requests avoid 403 responses.
Changes:
- Defaults fetched crate requests to the browser user agent.
- Preserves explicitly configured user agents.
- Adds coverage for both behaviours.
File summaries
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
8 tasks
The crates.io API now returns a 403 (Forbidden) response for all requests seemingly unless they use a browser user agent (i.e., `:default` and `:curl` fail). Checks using livecheck's `Crate` strategy have continued to work because the user agent fallback logic in `Strategy::page_content` tries the URL again using the `:browser` user agent (after failing with `:default`) and that works. However, this fallback logic will be deprecated in the near future and these checks will error instead (not to mention we don't need to be trying with a user agent we know will fail before using one that will work). This issue affects all of the `Crate` strategy checks, so this adds logic to the `Crates::find_versions` method to set a `:browser` user agent if one isn't set in the `livecheck` block. For the sake of simplicity, I haven't included this in the `match_data` return value as something that would be surfaced in debug and verbose JSON output. It's technically possible (and I prototyped it) but it requires some changes to the JSON structure and I wasn't convinced it was worth it. I may revisit it in the future but for now I'm treating it as an implementation detail.
samford
force-pushed
the
livecheck/crate-add-default-user-agent
branch
from
September 6, 2026 00:07
4487f09 to
d8b62ce
Compare
MikeMcQuaid
approved these changes
Sep 6, 2026
Comment on lines
+98
to
99
Member
There was a problem hiding this comment.
Not this PR: would be nice to refactor these APIs to stop passing around options hashes which can't really be typed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
brew benchmarkresults.brewcommands to reproduce the bug?brew lgtm(style, typechecking and tests) locally?The crates.io API now returns a 403 (Forbidden) response for all requests seemingly unless they use a browser user agent (i.e.,
:defaultand:curlfail). Checks using livecheck'sCratestrategy have continued to work because the user agent fallback logic inStrategy::page_contenttries the URL again using the:browseruser agent (after failing with:default) and that works. However, this fallback logic will be deprecated in the near future and these checks will error instead (not to mention we don't need to be trying with a user agent we know will fail before using one that will work).This issue affects all of the
Cratestrategy checks, so this adds logic to theCrates::find_versionsmethod to set a:browseruser agent if one isn't set in thelivecheckblock. For the sake of simplicity, I haven't included this in thematch_datareturn value as something that would be surfaced in debug and verbose JSON output. It's technically possible (and I prototyped it) but it requires some changes to the JSON structure and I wasn't convinced it was worth it. I may revisit it in the future but for now I'm treating it as an implementation detail.