{{ message }}
Repository navigation
Polish up logging #157
Description
In the current state, we make a new logger in every module that needs to log something. There is CrawleeLogFormatter, which handles logging in the console.
- our loggers should default to the "info" level
- ??? the log format for Apify platform may be different (doesn't it ingest JSON? probably not, but I'm not sure)
- in JS Crawlee, some classes accept loggers instead of creating them - we may want to revisit that
- printing extra key-value information should now be possible via
kwargsoflogging.infoand related methods - should we use that?
This will be a breaking change.
Activity
vdusek commentedon May 20, 2024
Also, a question that comes to mind: do we want to override the default Python logger formatting? considering Crawlee is a library
janbuchar commentedon May 20, 2024
And also a framework... It is not unusual for frameworks to define their own loggers. But if a client wants to override that, we should make it easy.
vdusek commentedon Nov 21, 2025
We should also review the debug logs to ensure they provide some value. Suppose a user switches from info to debug level. In that case, they should get some meaningful information (e.g., about sending this request, processing this response, storing this item into the dataset, ...) that could help them (currently it is quite random and maybe a bit of spam).