Repository navigation
Comparing changes
Open a pull request
base repository: ScrapeGraphAI/Scrapegraph-ai
base: v2.2.3
head repository: ScrapeGraphAI/Scrapegraph-ai
compare: v2.2.4-beta.1
- 12 commits
- 6 files changed
- 6 contributors
Commits on Aug 21, 2026
-
ci(release): 2.2.0-beta.7 [skip ci]
## [2.2.0-beta.7](v2.2.0-beta.6...v2.2.0-beta.7) (2026-08-21) ### Bug Fixes * resolve markdown links from the document URL ([875385b](875385b)) ### Docs * fix timeout documentation links ([4763fdc](4763fdc))
Configuration menu - View commit details
-
Copy full SHA for decfc6b - Browse repository at this point
Copy the full SHA decfc6bView commit details
Commits on Aug 23, 2026
-
fix(fetch): surface HTTP errors and missing content instead of answer…
…ing NA A page that could not be scraped as intended was indistinguishable from one that could. FetchNode's default path (ChromiumLoader -> ascrape_playwright) dropped the Response returned by page.goto(), so a 404, 403, 500, captcha wall or login redirect reached the LLM as ordinary content and the model answered "NA" with nothing in the logs to explain why. Reported in #1102, where en.wikipedia.org/wiki/Timpson_(company) 404s (the article is at Timpson_(retailer)) and the run still looked clean. Two deterministic, LLM-free guards, both warnings so existing behaviour is unchanged for anyone deliberately scraping error pages: - ChromiumLoader keeps the Response from every page.goto() call site (ascrape_playwright, ascrape_playwright_scroll, ascrape_with_js_support) and warns on status >= 400. This mirrors what the opt-in use_soup=True path in FetchNode has always done. - ParseNode warns when the parsed content contains none of the terms the user asked about — schema field names plus the significant words of the prompt. A 200 response can still reach the LLM without the requested data: content behind JavaScript that never rendered, a field inside a <script> blob the parser drops, or a document truncated beyond the model window. Zero matches is a deliberately conservative bar, so the warning stays quiet when the page simply phrases the answer differently. The graphs now pass their schema to ParseNode so it has the field names available. Verified against the URLs from the issue: the 404 now logs "Received HTTP 404 for .../Timpson_(company); the scraped content is likely an error page" before returning NA, while the corrected URL stays silent and answers 1865. Also drops three dead imports from smart_scraper_multi_batch_graph.py, which ruff blocks on now that the file is touched. Fixes #1102 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Configuration menu - View commit details
-
Copy full SHA for f91478e - Browse repository at this point
Copy the full SHA f91478eView commit details -
Merge pull request #1135 from ScrapeGraphAI/fix/1102-silent-error-pag…
…e-detection fix(fetch): surface HTTP errors and missing content instead of answering NA
Configuration menu - View commit details
-
Copy full SHA for be1c9a8 - Browse repository at this point
Copy the full SHA be1c9a8View commit details -
Configuration menu - View commit details
-
Copy full SHA for d59c3df - Browse repository at this point
Copy the full SHA d59c3dfView commit details
Commits on Aug 30, 2026
-
Configuration menu - View commit details
-
Copy full SHA for c21af20 - Browse repository at this point
Copy the full SHA c21af20View commit details
Commits on Sep 1, 2026
-
fix: 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not…
… the config file `_check_config_and_environ_for_telemetry_flag` checked that the environment variable existed and then read its value out of the config file: if os.environ.get("SCRAPEGRAPHAI_TELEMETRY_ENABLED") is not None: try: telemetry_enabled = config_obj.getboolean("DEFAULT", "telemetry_enabled") except Exception: pass With no `telemetry_enabled` key in `~/.scrapegraphai.conf`, `getboolean` raises, the bare `except` swallows it, and the flag keeps its default of `True`. So the opt-out documented in the README leaves telemetry on for anyone who has not also written a config file. Now parses the variable's own value, reusing `configparser`'s BOOLEAN_STATES so the environment variable and the config file accept the same spellings (true/false, yes/no, on/off, 1/0). An unparseable value logs a warning and leaves the flag alone rather than failing silently. Adds tests/test_telemetry_flag.py covering the config path, the environment path, precedence between them, and an unparseable value. Verified with SCRAPEGRAPHAI_TELEMETRY_ENABLED=false and no config key: before: True after: FalseConfiguration menu - View commit details
-
Copy full SHA for 8769c3b - Browse repository at this point
Copy the full SHA 8769c3bView commit details
Commits on Sep 7, 2026
-
Merge pull request #1140 from amirshahzadhashmi7145/fix/1121-add-gemi…
…ni-2.5-tokens fix(models): add Gemini 2.5 token limits
Configuration menu - View commit details
-
Copy full SHA for 035087b - Browse repository at this point
Copy the full SHA 035087bView commit details -
ci(release): 2.2.0-beta.9 [skip ci]
## [2.2.0-beta.9](v2.2.0-beta.8...v2.2.0-beta.9) (2026-09-07) ### Bug Fixes * **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](c21af20))
Configuration menu - View commit details
-
Copy full SHA for 3047ef8 - Browse repository at this point
Copy the full SHA 3047ef8View commit details -
Merge pull request #1141 from HKlabworks/fix/telemetry-env-var-opt-out
fix: 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file
Configuration menu - View commit details
-
Copy full SHA for 1cd076b - Browse repository at this point
Copy the full SHA 1cd076bView commit details -
ci(release): 2.2.0-beta.10 [skip ci]
## [2.2.0-beta.10](v2.2.0-beta.9...v2.2.0-beta.10) (2026-09-07) ### Bug Fixes * 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](8769c3b))
Configuration menu - View commit details
-
Copy full SHA for 0bb8bc9 - Browse repository at this point
Copy the full SHA 0bb8bc9View commit details -
Merge branch 'main' into pre/beta
Brings v2.2.1-v2.2.3 from main into pre/beta, including the proxy rotation fix (#1145). Conflicts resolved by keeping pre/beta's pre-release version in pyproject.toml and main's stable CHANGELOG, both of which semantic-release regenerates on the next release.
Configuration menu - View commit details
-
Copy full SHA for aab4b87 - Browse repository at this point
Copy the full SHA aab4b87View commit details -
ci(release): 2.2.4-beta.1 [skip ci]
## [2.2.4-beta.1](v2.2.3...v2.2.4-beta.1) (2026-09-07) ### Bug Fixes * 🐛 read SCRAPEGRAPHAI_TELEMETRY_ENABLED from the environment, not the config file ([8769c3b](8769c3b)) * **models:** add Gemini 2.5 token limits so they are not truncated to 8192 ([c21af20](c21af20)) * **fetch:** surface HTTP errors and missing content instead of answering NA ([f91478e](f91478e)), closes [#1102](#1102) [#1102](#1102) ### CI * **release:** 2.2.0-beta.10 [skip ci] ([0bb8bc9](0bb8bc9)) * **release:** 2.2.0-beta.7 [skip ci] ([decfc6b](decfc6b)) * **release:** 2.2.0-beta.8 [skip ci] ([d59c3df](d59c3df)), closes [#1102](#1102) [#1102](#1102) * **release:** 2.2.0-beta.9 [skip ci] ([3047ef8](3047ef8))
Configuration menu - View commit details
-
Copy full SHA for 8b3a97c - Browse repository at this point
Copy the full SHA 8b3a97cView commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v2.2.3...v2.2.4-beta.1
