I'm not sure what to do about the screenshot test, as I really want to confirm that the figure is sent to the browser, but browsers differ in how they show the resize grip, which causes te comparison to fail.

Can you mask the resize grip by drawing a white rectangle on top using imagemagic?

result = next(iter(done)).result()
E       playwright._impl._api_types.Error: Executable doesn't exist at /home/vsts/.cache/ms-playwright/chromium-1015/chrome-linux/chrome
E       ╔════════════════════════════════════════════════════════════╗
E       ║ Looks like Playwright was just installed or updated.       ║
E       ║ Please run the following command to download new browsers: ║
E       ║                                                            ║
E       ║     playwright install                                     ║
E       ║                                                            ║
E       ║ <3 Playwright Team                                         ║
E       ╚════════════════════════════════════════════════════════════╝

looks like we need some extra steps in the CI setup?

I'm not sure what to do about the screenshot test, as I really want to confirm that the figure is sent to the browser, but browsers differ in how they show the resize grip, which causes te comparison to fail.

Can you mask the resize grip by drawing a white rectangle on top using imagemagic?

Actually, I forgot we have access to which browser is running the test, so I could just have two test images... I'll see what the diff looks like.

So this is nearly working on GitHub Actions. The current failure is on macOS. For some reason, the canvas gets a border and it's black on Linux, but blue on macOS. There's also some slight difference in the resize grip size.

AppVeyor only fails one test, so it's likely a similar deal.

Azure passes on Linux, but fails on macOS and Windows, but on several more tests. I'm not sure why that is yet.

Current status:

1. Firefox renders the resize grip differently across platforms; I've added a simple edit with Pillow to smooth out those differences.
2. WebKit seems to be broken on Ubuntu (it generates an empty screenshot, and resizing fails), but the build that Playwright ships requires very old libraries, so I can't test it on Fedora.
3. WebKit fails on Windows, for interactive tests. The screenshot does appear to pass.
4. The macOS 14 build seems to get stuck for currently unknown reasons.

On all other platforms, Chromium and Firefox pass all tests.

On web-based projects it is common to do functional tests across the whole matrix of OS and web browser, but only take screenshots on one (linux chrome) to minimise the churn in test images caused by regular browser updates. It should be less of a problem here as the screenshot images are really those generated by Agg, but it is an option.

It might be worth adding some real plotted data to the screenshots, maybe just a line plot, to prove it works.

We did have a case where things broke on a specific browser #18142, so I would like to check all of them if possible. That was of course due to something that was eventually standardized and not likely to be something to worry about in the future, but who knows what else we might use later that is inconsistent.

I did limit the screenshot to just the canvas part that should come from us, to avoid most browser-specific UI (other than the resize grip which has been edited out.)