Repository navigation
[ENH]: Use a valid font cache from a read-only MPLCONFIGDIR instead of rebuilding in a temp dir #32246
Description
Problem
In reproducible or immutable environments — Nix store paths, read-only container layers, shared HPC installs — a natural pattern is to prewarm the font cache at build time and point MPLCONFIGDIR at the read-only result.
Today _get_config_or_cache_dir() gates on os.access(dir, os.W_OK) before anything else, so a directory containing a perfectly valid fontlist-*.json is rejected outright: matplotlib warns "... is not a writable directory", creates a temporary directory and rebuilds the font cache there on every process start — exactly the slow path the prewarmed cache was meant to avoid. On a small server (1 vCPU) that rebuild dominates our chart generation.
$ MPLCONFIGDIR=/tmp/warm python -c 'import matplotlib.font_manager' # prewarm
$ chmod -R a-w /tmp/warm
$ MPLCONFIGDIR=/tmp/warm python -c 'import matplotlib.pyplot'
Matplotlib created a temporary cache directory ... because there was an issue with the default pathReproduced on 3.11.1; the W_OK gate is unchanged on current main.
Proposed solution
When MPLCONFIGDIR is set explicitly, exists and contains a loadable font cache for the current version, use it read-only, falling back to a temporary directory only for writes that are actually attempted. Alternatively, split cache-dir resolution from config-dir resolution so a read-only prewarmed cache can be honored explicitly.
Activity
rcomer commentedon Aug 27, 2026
Hi @rokokol, thanks for the proposal. Is this something that has been a problem in one of your workflows, or is it more theoretical?
rokokol commentedon Aug 27, 2026
Real workflow, not theoretical — though your question made me measure it properly, and the measurement corrects one claim in my original post.
The setup. A small NixOS mail server generates one daily report with matplotlib, driven by a systemd unit. Everything the unit runs comes from the Nix store, which is read-only by construction. The natural shape is to build the font cache at image-build time and point
MPLCONFIGDIRat that store path. Since that path is read-only, the unit instead has to copy the prewarmed cache into a writable directory on every start:That copy exists only to work around the writability gate. It is also the kind of workaround that has to be explained to the next reader, so it is written down as a deviation in our repo with "matplotlib reading a valid cache from a read-only MPLCONFIGDIR" as the condition that retires it.
Minimal reproduction:
Two things here beyond the rebuild itself: the valid
fontlist-v3.11.0.jsonsitting in that directory is never consulted, and the advice — "set the MPLCONFIGDIR environment variable to a writable directory" — is given to someone who did setMPLCONFIGDIRexplicitly, about what the message calls "the default path".Correction to my original post. I wrote that the rebuild "dominates our chart generation". Measured, it does not, and I should not have said it without measuring. Wall clock for
import matplotlib.pyplot, best of three:So the runtime cost is real but modest — roughly +0.2 s per process, paid on every run. The friction I would actually like removed is structural rather than measured in seconds: in an immutable deployment every consumer must add a copy step, and the diagnostic points away from the cause.
If honoring a read-only
MPLCONFIGDIRis more invasive than it looks, a smaller change would still help a lot: whenMPLCONFIGDIRis set explicitly and holds a loadable cache, say so in the warning instead of describing it as an issue with the default path. Happy to attempt either patch if you can say which direction is acceptable.tacaswell commentedon Aug 28, 2026
My only concern is that we sort out what path to use in
__init__(and wrap it so it caches) but try to read it until we importmatplotlib.font_managerso if we were to switch over when we needed to write, it would violate an assumption we currently have which is the paths are cached the first time they are accessed (which can be during the initial import ofmatplotlib.__init__) and then stable through the rest of the process. I'm not sure we actually rely on that and always call the functions (so we have it cached exactly one place), but it could still be surprising to someone. IT also looks like this has been the behavior from ~2005 efa9ea6 .We also use the same cache directory as part of the latex processing (to cache the results of shelling out which can have big wins even in a single process) which is much more likely to be written to in any given process.
At some level it seems like a very fair expectation of a library that the directory it will use for caching is writable so that it can cache things. There are better tools (like overlay filesystems, bwrap, or containers) to discard writes by an un-trusted process.
There is also a question of if we want to treat "non existent" and "not fully usable" as the same.
I'm torn between the expectation that most of the users who end up pointing at a read-only folder do it on purpose (so we should not fall back to tmp but should just let the the writes fail when we try to write) or if most do it by accident and we should switch to tmp so latex is not as slow.
Sorry, I'm thinking through this as I'm typing, the TL;DR I could be convinced to drop the writable check so long as we are sure every place we do try to write will gracefully fail (which we should do anyway as the filesystem can always changed under us).
One very quick fix is that we missed an 'f' on an f-string! It looks like we gained different ways to specify the config/cache directory but we did not make sure the error messages were also updated.
rokokol commentedon Aug 28, 2026
Thanks — both constraints check out from my reading too:
get_configdir/get_cachedirare@_logged_cachedand are often first hit duringimport matplotlib, so swapping the directory at first write would break the "stable for the process lifetime" assumption; and the same cache dir carriestex.cache(texmanager) andvartexfonts(dviread), which are written far more often than the font list.I've opened #32252 for the message half: restoring the lost
f(it went missing in #30454, so the literal{configdir}ships since 3.11.0) and namingMPLCONFIGDIRwhen it is set instead of "the default path".On non-existent vs not-fully-usable: treating them differently seems right to me. A directory that doesn't exist and can't be created leaves the tmp fallback as the only sensible behavior. A directory that exists but is read-only is much more likely deliberate (immutable images, read-only containers), which matches your first branch: keep it and let individual writes fail gracefully.
A quick audit of the write sites against the "every write fails gracefully" condition:
font_manager.json_dumpalready catchesOSErrorand logs "Could not save font_manager cache" — the font path is graceful today, and with a valid cache present it never writes at all.TexManager.__new__callsmkdir(parents=True, exist_ok=True)ontex.cache, and_get_base_pathmkdirs hash subdirectories — these raise on a read-only parent, so usetex is where the graceful-failure work actually lives.dvireadonly passesMT_VARTEXFONTSdown to mktexpk, so failures surface through the existing subprocess error path.testing/compare.pywritestest_cache, test-suite only.Happy to take that as a follow-up PR if the direction is acceptable.
WeatherGod commentedon Aug 28, 2026