You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
How to config user_data_dir when combine crawlee playwright camoufox #1776
I initialized my project using Crawlee CLI with Playwright and Camoufox, and I am trying to configure user_data_dir for persistent login sessions.
Although the profile data is correctly created in the specified user_data_dir after logging in to Google, the session is not reused on subsequent runs. When I restart the crawler, it redirects to the Google login page instead of restoring the previous authenticated session.
fromcamoufoximportAsyncNewBrowserfromcrawleeimportRequestfromcrawlee._utils.contextimportensure_contextfromcrawlee.browsersimportPlaywrightBrowserPlugin, PlaywrightBrowserController, BrowserPoolfromcrawlee.crawlersimportPlaywrightCrawlerfromtyping_extensionsimportoverridefrom .constants.HandlerTypeimportHandlerTypefrom .routesimportrouterclassCamoufoxPlugin(PlaywrightBrowserPlugin):
"""Example browser plugin that uses Camoufox Browser, but otherwise keeps the functionality of PlaywrightBrowserPlugin."""def__init__(self, user_data_dir: str=None):
super().__init__()
self.user_data_dir=user_data_dir@ensure_context@overrideasyncdefnew_browser(self) ->PlaywrightBrowserController:
ifnotself._playwright:
raiseRuntimeError('Playwright browser plugin is not initialized.')
returnPlaywrightBrowserController(
browser=(awaitAsyncNewBrowser(self._playwright, persistent_context=True,
headless=False,
user_data_dir=self.user_data_dir,
)).browser,
max_open_pages_per_browser=1, # Increase, if camoufox can handle it in your use case.header_generator=None, # This turns off the crawlee header_generation. Camoufox has its own.,
)
asyncdefmain() ->None:
"""The crawler entry point."""crawler=PlaywrightCrawler(
max_request_retries=0,
max_requests_per_crawl=10,
request_handler=router,
fingerprint_generator=None,
browser_pool=BrowserPool(
plugins=[CamoufoxPlugin('xxx')]),
)
awaitcrawler.run(
[
Request.from_url(url='https://accounts.google.com/', label=HandlerType.GOOGLE_LOGIN, user_data={
'email': 'xxx',
'password': 'xxx'
})
]
)
Hello, thanks for reporting this. We will look into this. Probably it has something to do with the default browser context that is already opened by Camoufox, but we do not use it.
Activity
Pijukatel commentedon Mar 5, 2026
Hello, thanks for reporting this. We will look into this. Probably it has something to do with the default browser context that is already opened by Camoufox, but we do not use it.
fix: Re-use pre-existing browser context in `PlaywrightBrowserControl…
Pijukatel commentedon Mar 6, 2026
@ninhthuanntnt there should be a fix on the latest beta release https://pypi.org/project/crawlee/1.4.1b7/
Your code snippet should work unchanged, and the Camoufox will use the persistent context.
We will include the fix in the upcoming minor release as well.