Comparing v2.2.3...v2.2.4-beta.1 · ScrapeGraphAI/Scrapegraph-ai · GitHub
Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: ScrapeGraphAI/Scrapegraph-ai
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v2.2.3
Choose a base ref
...
head repository: ScrapeGraphAI/Scrapegraph-ai
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v2.2.4-beta.1
Choose a head ref
  • 12 commits
  • 6 files changed
  • 6 contributors

Commits on Aug 21, 2026

  1. ci(release): 2.2.0-beta.7 [skip ci]

    ## [2.2.0-beta.7](v2.2.0-beta.6...v2.2.0-beta.7) (2026-08-21)
    
    ### Bug Fixes
    
    * resolve markdown links from the document URL ([875385b](875385b))
    
    ### Docs
    
    * fix timeout documentation links ([4763fdc](4763fdc))
    semantic-release-bot committed Aug 21, 2026
    Configuration menu
    Copy the full SHA
    decfc6b View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2026

  1. fix(fetch): surface HTTP errors and missing content instead of answer…

    …ing NA
    
    A page that could not be scraped as intended was indistinguishable from one
    that could. FetchNode's default path (ChromiumLoader -> ascrape_playwright)
    dropped the Response returned by page.goto(), so a 404, 403, 500, captcha wall
    or login redirect reached the LLM as ordinary content and the model answered
    "NA" with nothing in the logs to explain why. Reported in #1102, where
    en.wikipedia.org/wiki/Timpson_(company) 404s (the article is at
    Timpson_(retailer)) and the run still looked clean.
    
    Two deterministic, LLM-free guards, both warnings so existing behaviour is
    unchanged for anyone deliberately scraping error pages:
    
    - ChromiumLoader keeps the Response from every page.goto() call site
      (ascrape_playwright, ascrape_playwright_scroll, ascrape_with_js_support) and
      warns on status >= 400. This mirrors what the opt-in use_soup=True path in
      FetchNode has always done.
    - ParseNode warns when the parsed content contains none of the terms the user
      asked about — schema field names plus the significant words of the prompt.
      A 200 response can still reach the LLM without the requested data: content
      behind JavaScript that never rendered, a field inside a <script> blob the
      parser drops, or a document truncated beyond the model window. Zero matches
      is a deliberately conservative bar, so the warning stays quiet when the page
      simply phrases the answer differently. The graphs now pass their schema to
      ParseNode so it has the field names available.
    
    Verified against the URLs from the issue: the 404 now logs "Received HTTP 404
    for .../Timpson_(company); the scraped content is likely an error page" before
    returning NA, while the corrected URL stays silent and answers 1865.
    
    Also drops three dead imports from smart_scraper_multi_batch_graph.py, which
    ruff blocks on now that the file is touched.
    
    Fixes #1102
    
    Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
    VinciGit00 and claude committed Aug 23, 2026
    Configuration menu
    Copy the full SHA
    f91478e View commit details
    Browse the repository at this point in the history
  2. Merge pull request #1135 from ScrapeGraphAI/fix/1102-silent-error-pag…

    …e-detection
    
    fix(fetch): surface HTTP errors and missing content instead of answering NA
    VinciGit00 authored Aug 23, 2026
    Configuration menu
    Copy the full SHA
    be1c9a8 View commit details
    Browse the repository at this point in the history
  3. ci(release): 2.2.0-beta.8 [skip ci]

    ## [2.2.0-beta.8](v2.2.0-beta.7...v2.2.0-beta.8) (2026-08-23)
    
    ### Bug Fixes
    
    * **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](f91478e)), closes [#1102](#1102) [#1102](#1102)
    semantic-release-bot committed Aug 23, 2026
    Configuration menu
    Copy the full SHA
    d59c3df View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2026

  1. Configuration menu
    Copy the full SHA
    c21af20 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2026

  1. fix: 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not…

    … the config file
    
    `_check_config_and_environ_for_telemetry_flag` checked that the environment
    variable existed and then read its value out of the config file:
    
        if os.environ.get("SCRAPEGRAPHAI_TELEMETRY_ENABLED") is not None:
            try:
                telemetry_enabled = config_obj.getboolean("DEFAULT", "telemetry_enabled")
            except Exception:
                pass
    
    With no `telemetry_enabled` key in `~/.scrapegraphai.conf`, `getboolean` raises,
    the bare `except` swallows it, and the flag keeps its default of `True`. So the
    opt-out documented in the README leaves telemetry on for anyone who has not also
    written a config file.
    
    Now parses the variable's own value, reusing `configparser`'s BOOLEAN_STATES so the
    environment variable and the config file accept the same spellings (true/false,
    yes/no, on/off, 1/0). An unparseable value logs a warning and leaves the flag alone
    rather than failing silently.
    
    Adds tests/test_telemetry_flag.py covering the config path, the environment path,
    precedence between them, and an unparseable value.
    
    Verified with SCRAPEGRAPHAI_TELEMETRY_ENABLED=false and no config key:
      before: True   after: False
    HKlabworks committed Sep 1, 2026
    Configuration menu
    Copy the full SHA
    8769c3b View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2026

  1. Merge pull request #1140 from amirshahzadhashmi7145/fix/1121-add-gemi…

    …ni-2.5-tokens
    
    fix(models): add Gemini 2.5 token limits
    VinciGit00 authored Sep 7, 2026
    Configuration menu
    Copy the full SHA
    035087b View commit details
    Browse the repository at this point in the history
  2. ci(release): 2.2.0-beta.9 [skip ci]

    ## [2.2.0-beta.9](v2.2.0-beta.8...v2.2.0-beta.9) (2026-09-07)
    
    ### Bug Fixes
    
    * **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](c21af20))
    semantic-release-bot committed Sep 7, 2026
    Configuration menu
    Copy the full SHA
    3047ef8 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #1141 from HKlabworks/fix/telemetry-env-var-opt-out

    fix: 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file
    VinciGit00 authored Sep 7, 2026
    Configuration menu
    Copy the full SHA
    1cd076b View commit details
    Browse the repository at this point in the history
  4. ci(release): 2.2.0-beta.10 [skip ci]

    ## [2.2.0-beta.10](v2.2.0-beta.9...v2.2.0-beta.10) (2026-09-07)
    
    ### Bug Fixes
    
    * 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](8769c3b))
    semantic-release-bot committed Sep 7, 2026
    Configuration menu
    Copy the full SHA
    0bb8bc9 View commit details
    Browse the repository at this point in the history
  5. Merge branch 'main' into pre/beta

    Brings v2.2.1-v2.2.3 from main into pre/beta, including the proxy
    rotation fix (#1145). Conflicts resolved by keeping pre/beta's
    pre-release version in pyproject.toml and main's stable CHANGELOG,
    both of which semantic-release regenerates on the next release.
    VinciGit00 committed Sep 7, 2026
    Configuration menu
    Copy the full SHA
    aab4b87 View commit details
    Browse the repository at this point in the history
  6. ci(release): 2.2.4-beta.1 [skip ci]

    ## [2.2.4-beta.1](v2.2.3...v2.2.4-beta.1) (2026-09-07)
    
    ### Bug Fixes
    
    * 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](8769c3b))
    * **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](c21af20))
    * **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](f91478e)), closes [#1102](#1102) [#1102](#1102)
    
    ### CI
    
    * **release:** 2.2.0-beta.10 [skip ci] ([0bb8bc9](0bb8bc9))
    * **release:** 2.2.0-beta.7 [skip ci] ([decfc6b](decfc6b))
    * **release:** 2.2.0-beta.8 [skip ci] ([d59c3df](d59c3df)), closes [#1102](#1102) [#1102](#1102)
    * **release:** 2.2.0-beta.9 [skip ci] ([3047ef8](3047ef8))
    semantic-release-bot committed Sep 7, 2026
    Configuration menu
    Copy the full SHA
    8b3a97c View commit details
    Browse the repository at this point in the history
Loading