What is this PR for?

Linux Python CI installs committed dependency locks to avoid resolving dependencies on every run. However, these locks are refreshed manually, so newer compatible dependencies remain untested and upstream breaking changes can go unnoticed.

This PR adds weekly dependency refreshes within the supported Python and backend versions. It runs the existing backend and Selenium tests against the candidate lock and proposes a draft PR only when the tests pass and dependencies have changed. Failures leave the committed lock unchanged. Regular CI continues using the committed lock, with no automatic merging.

The shared Python, TensorFlow and PyFlink environments are managed through pixi.toml and pixi.lock. Regular CI and refresh validation reuse the same test definitions.

backend-tests.yml and selenium-integration.yml extract existing jobs from core.yml and frontend.yml so regular CI and dependency refreshes can reuse the same test definitions.

Test commands and matrices only need to be maintained in one place, and dependency refreshes do not need to run unrelated jobs such as Windows builds, npm audit and Playwright tests.

What type of PR is it?

Improvement

Todos

• Complete backend and Selenium validation on GitHub Actions before merge.

What is the Jira issue?

• https://issues.apache.org/jira/browse/ZEPPELIN-6739

How should this be tested?

1. Run the core and frontend workflows. Verify that the existing backend and Selenium tests run using the committed lock without resolving dependencies.
2. Manually run Python dependency refresh on a test fork. Verify that it generates a candidate lock and runs both validation workflows using that candidate.
3. On a disposable branch, force a validation test to fail and rerun the refresh. Verify that the failure is visible in Actions.

Manual execution requires the workflow to exist on the fork’s default branch. Fork runs validate candidate generation and testing; actual draft-PR publication is restricted to apache/zeppelin master.

Questions:

• Does the license files need to update? No.
• Is there breaking changes for older versions? No.
• Does this needs documentation? Yes, updated in testing/README.md.