Commit history

1. Commits on May 2, 2025

   • feat: add `on_skipped_request` decorator, to process links skipped according to `robots.txt` rules (#1166)

     Show description for bd16f14
     

     4 peopleauthored
     bd16f14View commit details

     Copy full SHA for bd16f14

     View code at this pointBrowse repository at this point
2. Commits on Apr 24, 2025

   • feat: add `respect_robots_txt_file` option (#1162)

     Show description for c23f365
     

     4 peopleauthored
     c23f365View commit details

     Copy full SHA for c23f365

     View code at this pointBrowse repository at this point
3. Commits on Apr 10, 2025

   • docs: add example resume a crawl (#1143)

     Show description for 40edf70
     Mantisus

     authored
     40edf70View commit details

     Copy full SHA for 40edf70

     View code at this pointBrowse repository at this point
4. Commits on Apr 7, 2025

   • feat: Add `ErrorSnapshotter` to `ErrorTracker` (#1125)

     Show description for 9666092
     Pijukatel

     authored
     9666092View commit details

     Copy full SHA for 9666092

     View code at this pointBrowse repository at this point
5. Commits on Apr 3, 2025

   • feat: Add `requests` argument to `EnqueueLinksFunction` (#1024)

     Show description for fc8444c
     Pijukatel

     authored
     fc8444cView commit details

     Copy full SHA for fc8444c

     View code at this pointBrowse repository at this point

   • docs: Add "Run on Apify" button (#1127)

     Show description for e7ebda7
     vdusek

     authored
     e7ebda7View commit details

     Copy full SHA for e7ebda7

     View code at this pointBrowse repository at this point
6. Commits on Mar 18, 2025

   • feat: Add `statistics_log_format` parameter to `BasicCrawler` (#1061)

     Show description for 635ae4a
     Mantisus

     authored
     635ae4aView commit details

     Copy full SHA for 635ae4a

     View code at this pointBrowse repository at this point
7. Commits on Mar 3, 2025

   • chore: upgrade to Ruff v0.9 (#1026)

     Show description for 55e287f
     vdusek

     authored
     55e287fView commit details

     Copy full SHA for 55e287f

     View code at this pointBrowse repository at this point
8. Commits on Feb 25, 2025

   • refactor!: Replace enum with literal for `EnqueueStrategy` (#1019)

     Show description for d2481ef
     vdusek

     authored
     d2481efView commit details

     Copy full SHA for d2481ef

     View code at this pointBrowse repository at this point
9. Commits on Feb 18, 2025

   • feat: Add adaptive context helpers (#964)

     Show description for e248f17
     

     Pijukatel

     and

     janbuchar

     authored
     e248f17View commit details

     Copy full SHA for e248f17

     View code at this pointBrowse repository at this point
10. Commits on Feb 11, 2025

    • docs: Set line length to docs related code to 90 (#973)

      Show description for a2e8496
      Pijukatel

      authored
      a2e8496View commit details

      Copy full SHA for a2e8496

      View code at this pointBrowse repository at this point
11. Commits on Feb 7, 2025

    • feat: Add AdaptivePlaywrightCrawler (#872)

      Show description for 5ba70b6
      

      Pijukatel

      and

      janbuchar

      authored
      5ba70b6View commit details

      Copy full SHA for 5ba70b6

      View code at this pointBrowse repository at this point
12. Commits on Feb 5, 2025

    • feat: Integrate browserforge fingerprints (#829)

      Show description for 2b156b4
      

      3 peopleauthored
      2b156b4View commit details

      Copy full SHA for 2b156b4

      View code at this pointBrowse repository at this point
13. Commits on Jan 23, 2025

    • feat: add `block_requests` helper for `PlaywrightCrawler` (#919)

      Show description for 1030459
      

      Mantisus

      and

      vdusek

      authored
      1030459View commit details

      Copy full SHA for 1030459

      View code at this pointBrowse repository at this point
14. Commits on Jan 22, 2025

    • feat: Add keep_alive flag to `crawler.__init__` (#921)

      Show description for 7a82d0c
      Pijukatel

      authored
      7a82d0cView commit details

      Copy full SHA for 7a82d0c

      View code at this pointBrowse repository at this point
15. Commits on Jan 17, 2025

    • docs: Fix Playwright + Camoufox code example (#915)

      Show description for 5fc027a
      vdusek

      authored
      5fc027aView commit details

      Copy full SHA for 5fc027a

      View code at this pointBrowse repository at this point
16. Commits on Dec 20, 2024

    • refactor!: Update the crawlers & storage clients structure (#828)

      Show description for 0ba04d1
      vdusek

      authored
      0ba04d1View commit details

      Copy full SHA for 0ba04d1

      View code at this pointBrowse repository at this point

    • refactor!: Rename PlaywrightCrawler kwargs: browser_options, page_options (#831)

      Show description for ffc6048
      

      Pijukatel

      and

      vdusek

      authored
      ffc6048View commit details

      Copy full SHA for ffc6048

      View code at this pointBrowse repository at this point
17. Commits on Dec 19, 2024

    • refactor!: Rename PwPreNavContext to PwPreNavCrawlingContext (#827)

      Show description for 84b61a3
      vdusek

      authored
      84b61a3View commit details

      Copy full SHA for 84b61a3

      View code at this pointBrowse repository at this point
18. Commits on Dec 18, 2024

    • feat: Add stop method to BasicCrawler (#807)

      Show description for 6d01af4
      

      Pijukatel

      and

      vdusek

      authored
      6d01af4View commit details

      Copy full SHA for 6d01af4

      View code at this pointBrowse repository at this point
19. Commits on Dec 12, 2024

    • feat: Add example of how to integrate Camoufox into PlaywrightCrawler (#789)

      Show description for 246cfc4
      Pijukatel

      authored
      246cfc4View commit details

      Copy full SHA for 246cfc4

      View code at this pointBrowse repository at this point

    • feat: Add pre-navigation hooks router to AbstractHttpCrawler (#791)

      Show description for 0f23205
      

      Pijukatel

      and

      janbuchar

      authored
      0f23205View commit details

      Copy full SHA for 0f23205

      View code at this pointBrowse repository at this point
20. Commits on Nov 12, 2024

    • fix: Fix serialization payload in Request. Fix Docs for Post Request (#683)

      Show description for e8b4d2d
      

      Mantisus

      and

      vdusek

      authored
      e8b4d2dView commit details

      Copy full SHA for e8b4d2d

      View code at this pointBrowse repository at this point
21. Commits on Nov 1, 2024

    • chore!: Remove Request.query_params field (#639)

      Show description for 6ec0ec4
      vdusek

      authored
      6ec0ec4View commit details

      Copy full SHA for 6ec0ec4

      View code at this pointBrowse repository at this point
22. Commits on Oct 31, 2024

    • feat: Split and add extra configuration to export_data method (#580)

      Show description for 6751635
      

      6 peopleauthored
      6751635View commit details

      Copy full SHA for 6751635

      View code at this pointBrowse repository at this point
23. Commits on Oct 30, 2024

    • feat: Add `pre_navigation_hooks` to `PlaywrightCrawler` (#631)

      Show description for 5dd5b60
      Prathamesh010

      authored
      5dd5b60View commit details

      Copy full SHA for 5dd5b60

      View code at this pointBrowse repository at this point
24. Commits on Oct 24, 2024

    • fix!: merge payload and data fields of Request (#542)

      Show description for d06fcef
      vdusek

      authored
      d06fcefView commit details

      Copy full SHA for d06fcef

      View code at this pointBrowse repository at this point
25. Commits on Aug 23, 2024

    • refactor!: declare private and public interface (#456)

      Show description for d6738df
      vdusek

      authored
      d6738dfView commit details

      Copy full SHA for d6738df

      View code at this pointBrowse repository at this point

    • docs: move code samples to separate files (#457)

      Show description for 1c1ec64
      vdusek

      authored
      1c1ec64View commit details

      Copy full SHA for 1c1ec64

      View code at this pointBrowse repository at this point
26. Commits on Aug 22, 2024

    • feat: add support for filling a web form (#453)

      Show description for 5a125b4
      vdusek

      authored
      5a125b4View commit details

      Copy full SHA for 5a125b4

      View code at this pointBrowse repository at this point

    • docs: add links to API doc in suitable places (#449)

      Show description for ada0990
      vdusek

      authored
      ada0990View commit details

      Copy full SHA for ada0990

      View code at this pointBrowse repository at this point
27. Commits on Aug 7, 2024

    • feat: Implement ParselCrawler that adds support for Parsel (#348)

      Show description for a3832e5
      

      asymness

      and

      janbuchar

      authored
      a3832e5View commit details

      Copy full SHA for a3832e5

      View code at this pointBrowse repository at this point
28. Commits on Jul 9, 2024

    • doc: improve home page and playwright crawler example (#282)

      Show description for 640a4a4
      vdusek

      authored
      640a4a4View commit details

      Copy full SHA for 640a4a4

      View code at this pointBrowse repository at this point

    • doc: improve guide proxy management (#278)

      vdusek

      authored
      e927321View commit details

      Copy full SHA for e927321

      View code at this pointBrowse repository at this point
29. Commits on Jul 2, 2024

    • doc: finish the examples section (#261)

      Show description for 5819610
      vdusek

      authored
      5819610View commit details

      Copy full SHA for 5819610

      View code at this pointBrowse repository at this point