We can use https://github.com/actions/stale now. Example implementation in https://github.com/sunpy/sunpy/blob/main/.github/workflows/stale_bot.yml . Have to make sure our settings still comply with https://docs.astropy.org/en/latest/development/maintainers/maintainer_workflow.html#stale-policies (and should also update the doc).

Open question: Do we need to remove the "keep-open" label first before implementing this?